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Abstract 


The  goal  of  this  study  is  to  offer  senior  decision-makers  a  useful  framework  to 
evaluate,  articulate,  and  recommend  modifications  of  the  government’s  bid  protest 
policy  to  improve  procurement  outcomes.  Most  countries  allow  disappointed  bidders 
to  protest  public  procurement  decisions  as  an  oversight  mechanism  to  minimize 
fraud  and  errors  by  procurement  officials.  The  expectation  is  that  allowing  bid 
protests  increases  transparency  and  accountability  and  encourages  competition, 
reducing  the  cost  of  public  purchases.  A  key  contribution  of  this  study  is  to  add  an 
economics  dimension  to  a  bid  protest  process  that  is  mostly  thought  of  in  legal 
terms.  The  economics  approach  suggests  firms  are  likely  to  use  the  protest  process 
strategically  to  improve  their  competitive  bargaining  position  and  will  be  equally 
strategic  in  their  decision  to  protest  an  award.  The  conclusion  is  that  a  firm’s 
responsibility  to  its  shareholders  can  lead  it  to  undertake  protests  for  reasons 
substantially  different  from,  and  fundamentally  opposed  to,  the  government’s 
objectives.  The  economics  approach  suggests  minimizing  the  risk  of  protests  that 
result  in  cost  overruns,  schedule  delays,  and  performance  gaps,  while  preserving 
the  benefits  offered  by  protests  to  promote  competition  and  ensure  the  integrity, 
transparency,  and  accountability  of  the  procurement  process.  Adopting  an 
economics  perspective  reveals  two  other  crucial  insights:  firms  can  exploit  protests 
to  extract  concessions  (Fed  Mail),  and  risk-averse  officials  can  overreact  in  attempts 
to  achieve  protest-proof  procurements  (Buy-offs).  In  recognizing  costs  as  well  as 
benefits  of  a  protest  system,  the  economics  approach  invites  a  review  of  alternative 
portfolios  of  governance  mechanisms  that  complement  bid  protests  (internal  audits, 
external  audits,  independent  investigations,  alternative  dispute  resolution,  increased 
training  and  incentives,  etc.),  to  improve  procurement  outcomes. 

Keywords:  Bid  protest,  transaction  costs,  Principal-Agent  Model 
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Executive  Summary 


The  goal  of  this  study  was  to  offer  senior  decision-makers  a  useful  framework 
and  perspective  to  evaluate,  articulate,  and  recommend  modifications  of  the 
government’s  bid  protest  policy  to  improve  procurement  outcomes.A  key  contribution 
is  to  add  an  economics  dimension  to  a  process  that  is  mostly  thought  of  in  legal 
terms. 


The  legal  and  regulatory  literature  that  underpins  the  current  protest  system 
reflects  two  fundamental  beliefs.  Protests  (a)  deter  fraud  and  errors  in  government 
procurements  and  (b)  promote  competition.  This  study  explores  the  first  premise 
through  the  economist’s  “principal-agent”  framework  and  explores  the  second  with 
the  aid  of  a  representative  bidder  model.  The  aim  is  to  examine  opportunities  to 
improve  the  efficiency  and  effectiveness  of  the  protest  process  in  order  to  enhance 
performance,  cost,  and  schedule  outcomes  of  defense  acquisition  investments. 

In  economics,  there  is  no  such  thing  as  a  “free  lunch.”  Operating  a  protest 
system  is  costly.  The  substantial  costs  of  protests  to  the  military  departments  are  a 
significant  challenge.  This  is  reflected  in  the  U.S.  Air  Force’s  recent  experience 
selecting  a  vendor  for  the  KC-X  aerial  tanker. 

Along  with  the  well-documented  benefits  of  allowing  firms  to  protest  public 
procurements  come  significant  opportunity  and  transaction  costs.  The  goal  can  be 
stated  as  follows:  Reduce  the  risk  of  protests  of  critical  acquisitions  that  result  in 
significant  cost  overruns,  schedule  delays,  and  performance  gaps,  while  preserving 
the  benefits  offered  by  protests  to  promote  competition  and  ensure  the  integrity, 
transparency,  and  accountability  of  the  procurement  process. 

Adopting  an  economics  perspective  reveals  two  other  crucial  insights:  first, 
firms  can  exploit  protests  as  a  strategic  bargaining  tool;  second,  governments  can 
overreact  in  attempts  to  achieve  protest-proof  procurements.  These  hidden  costs 
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significantly  impact  acquisitions.  Combined,  they  substantially  increase  risks  of 
delays  and  cost-overruns. 

Fortunately,  fighting  agency  errors  and  corruption  and  promoting  competition 
can  be  done  in  other  ways.  The  economics  approach  encourages  senior  leadership 
to  identify  and  evaluate  alternatives  to  protests  that  could  achieve  similar  results  at  a 
lower  cost.  Acknowledging  the  substantial  costs  of  operating  a  protest  system  to 
govern  procurements  and  encourage  competition  is  one  of  the  key  insights  of  the 
study:  There  exists  an  “optimum”  rate  of  protests.  The  optimum  rate  of  protests 
offers  the  most  efficient  and  effective  way  to  encourage  competition  and  ensure  the 
integrity  of  the  acquisition  process  while  guaranteeing  the  best  possible  outcomes 
for  our  troops  and  taxpayers. 

Although  it  is  not  easy  to  locate  this  optimum,  it  is  possible  to  identify  and 
evaluate  policies  that  approach  the  optimum.  One  important  implication  is  that  a 
simple,  often-stated  department  objective  such  as  “reducing  the  rate  of  successful 
protests”  is  not  always  appropriate.  Two  additional  factors  need  to  be  considered. 
The  first  is  to  identify  where  the  department  is  relative  to  the  optimum.  The  second 
involves  a  careful  consideration  of  how  the  department  might  choose  to  reduce 
protests.  By  recognizing  the  costs  as  well  as  the  benefits  of  a  protest  system,  the 
economics  approach  invites  a  review  of  portfolios  of  governance  mechanisms 
(internal  audits,  external  audits,  independent  investigations,  ADR,  integrity  pacts, 
training,  incentives,  etc.)  that  can  substitute  for  (or  complement)  protests  to  improve 
procurement  outcomes. 

If  a  military  department  finds  itself  in  a  situation  in  which  it  experiences 
contract  challenges  above  the  optimum  rate,  then  the  burden  of  protests  outweighs 
the  benefits  and  reducing  the  rate  of  protests  is  appropriate.  This  can  be 
accomplished  in  two  ways:  by  reducing  the  expected  benefits  of  a  protest  to  a 
“disappointed  bidder”  (including  by  enabling  the  protester  to  achieve  desired 
outcomes  through  other  means)  or  by  increasing  the  expected  costs.  This  is  a 
crucial  distinction  emphasized  throughout  this  study. 
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If  a  military  department  is  below  the  optimum  rate,  then  the  benefits  of 
protests  outweigh  the  extra  (opportunity  and  transaction)  costs.  In  this  range,  while 
reducing  protests  might  appear  to  have  positive  short-run  impacts,  it  could  be  at  the 
expense  of  long-term  benefits  such  as  preserving  the  integrity  of  the  procurement 
system  and  encouraging  future  competition.  Remarkably,  this  report  reveals  two 
important  cases  that  demonstrate  that  even  in  this  range,  it  is  possible  to  reduce 
protests  and  improve  long-term  outcomes.  The  key  is  to  carefully  consider  how  the 
department  reduces  protests. 

This  study  captures  insights  from  the  protest  literature,  offers  a  brief  overview 
of  the  primary  U.S.  protest  process  administered  by  the  Government  Accountability 
Office  (GAO),  and  discusses  alternative  governance  mechanisms.  Woven 
throughout  are  data,  examples,  and  survey  results.  Two  economic  frameworks  are 
introduced — the  principal-agent  model  (agency  theory)  and  a  representative  bidder 
model.  These  frameworks  reveal  two  sets  of  potential  change  candidates  to  reduce 
the  risk  of  protests:  factors  that  reduce  the  expected  benefits  to  a  losing  bidder  of  a 
protest  and  factors  that  raise  the  expected  costs. 
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I. 


Introduction 


If  those  affected  by  the  breach  of  rules  cannot  protest  in  a  meaningful  way, 
the  rules  have  no  teeth,  and  competition  is  stifled.  Without  the  constraints  of 
bid  protests,  government  contracts  will  be  let  based  on  favoritism... and 
bribery — as  they  were  before  the  system  was  initiated.  (Weckstein  &  Love, 
1995) 

This  study  examines  opportunities  to  improve  the  efficiency  and  effectiveness 
of  the  protest  process  in  order  to  enhance  the  performance,  cost,  and  schedule 
outcomes  of  defense  acquisition  investments.  The  goal  is  to  offer  senior  decision¬ 
makers  a  useful  framework  and  perspective  that  allows  them  to  evaluate,  articulate, 
and  prescribe  protest  policy  recommendations  that  improve  procurement  outcomes. 

As  expressed  in  the  introductory  quotation,  the  legal  and  regulatory  literature 
that  underpins  the  current  protest  system  reflects  (a)  fundamental  doubts  about 
whether  government  officials  will  exercise  proper  discretion  to  the  benefit  of 
taxpayers,  and  (b)  a  belief  that  protests  promote  competition  to  the  benefit  of  our 
troops  and  taxpayers  (for  example,  see  Arrowsmith,  Linarelli,  &  Wallace,  2000; 
Gordon,  2006;  Metzger  &  Lyons,  2007;  Troff,  2005).  This  study  explores  the  first 
premise  through  the  economist’s  “principal-agent”  framework  and  explores  the 
second  with  the  aid  of  a  representative  bidder  model. 

According  to  the  Federal  Acquisition  Regulation  (FAR),  a  protest  is  defined 
as: 


a  written  objection  by  an  interested  party  to  any  of  the  following:  (1 )  A 
solicitation  ...  by  an  agency  for  offers  for  a  contract  for  the  procurement  of 
property  or  services,  (2)  The  cancellation  of  the  solicitation  ...  (3)  An  award  or 
proposed  award  of  the  contract,  (4)  A  termination  or  cancellation  of  an  award 
of  the  contract.  (FAR,  2005,  Subpart  33.101;  see  also  U.S.C.  31  §  3551  [1]) 
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A  distribution  of  the  relative  frequency  of  protests  over  these  four  categories 
would  reveal  that  the  majority  of  protests  involve  “an  award  or  proposed  award  of  the 
contract,”  which  is  the  primary  focus  of  this  study.1 

An  underlying  hypothesis  in  the  literature  is  that  protests  play  an  important 
role  as  a  credible,  decentralized  oversight  mechanism  for  the  acquisition  process, 
and  that  this  helps  deter  procurement  errors,  fraud,  and  favoritism,  which  in  turn 
makes  firms  more  willing  and  able  to  compete  for  government  contracts  since  they 
are  less  likely  to  be  excluded  for  spurious  reasons.  The  presumption  that  follows  is 
that  since  competitive  markets  offer  better  performance,  faster  schedules,  and 
cheaper  costs,  a  bid  protest  process  that  increases  competition  can  improve 
procurement  outcomes.2  A  key  pillar  of  the  federal  government’s  protest  process — 
the  Competition  in  Contracting  Act  (CICA) — is  largely  based  on  this  premise.3 
However,  the  protest  process  can  also  be  abused  to  restrict  competition.4 


1  This  study  has  this  primary  purpose  in  part  because  this  category  involves  the  most  time  and 
financial  investments  by  the  agency,  and  in  part  because  bid  protests  at  this  stage  have  a  significant 
impact  on  the  acquisition  process. 

2  The  economist’s  “transaction  costs”  literature  questions  this  assumption,  especially  when  ex-ante 
competition  leads  to  ex-post  monopoly  power.  The  risk  is  that  a  winning  “foot-in-the-door  strategy” 
later  leads  to  a  “hold-up,”  in  which  the  winning  firm  makes  up  any  losses  from  its  low  initial  bid 
through  change  orders,  etc.  (see  Melese,  Franck,  Angelis,  &  Dillard,  2007)  A  related  piece  of 
literature  on  “contestable  markets”  argues  that  low  entry  barriers  (the  degree  of  asset  specificity,  the 
ease  of  re-competing  the  contract,  etc.)  are  more  important  than  the  number  of  competitors  in 
obtaining  competitive  market  outcomes  (Baumol,  Panzar,  &  Willig,  1982).  Because  a  stated  goal  of 
the  protest  system  is  to  increase  vendor  participation  in  procurements  (Arrowsmith  et.  al. ,  2000),  this 
could  be  an  interesting  avenue  for  future  research. 

3  Along  with  the  Federal  Acquisition  Regulation  (FAR  Parts  5,  10,  12-15,  and  33),  the  Tucker  Act,  Title 

28,  Section  1491  of  the  U.S.  Code,  Executive  Order  No.  12979,  Agency  Procurement  Protests,  and 
various  case  law  precedents,  the  Competition  in  Contracting  Act  (CICA)  of  1984  (Title  31  of  the  U.S. 
Code  Sections  3551-3556)  is  a  key  pillar  of  the  protest  process.  The  CICA  completed  the  foundation 
for  the  modern  bid  protest  structure.  The  CICA  reflects  a  strong  presumption  that  government 
purchasing  agencies  should  use  competitive  procurement  techniques  to  increase  opportunities  for 
firms  to  compete  for  contract  awards.  Today,  one  of  the  guiding  principles  of  CICA  and  the  Federal 
Acquisition  Regulation  (FAR)  is  to  promote  competition  for  government  contracts.  The  CICA  laid  the 
foundation  for  the  existing  bid  protest  system.  In  the  CICA,  Congress  expressly  codified  the  authority 
of  the  Government  Accountability  Office  (GAO)  to  adjudicate  procurement  protests  by  disappointed 
bidders  or  prospective  bidders.  Congress  directed  that  “to  the  maximum  extent  practicable,  the 
Comptroller  General  (at  GAO)  shall  provide  for  the  inexpensive  and  expeditious  resolution  of 
protests”  (31  U.S.C.  §  3554(a)).  The  CICA  gives  the  GAO  authority  over  bid  protests  as  an 
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A  fundamental  challenge  identified  by  the  procuring  agencies  is  the  significant 
costs  that  protests  impose  on  the  department.  This  is  reflected  in  the  recent 
selection  of  a  vendor  for  the  new  KC-X  aerial  refueling  tanker.* * * 4 5  The  Defense 
Department’s  aim  is  to  reduce  the  risk  of  protests  of  critical  acquisitions  that  result  in 
significant  cost  overruns,  schedule  delays,  and  performance  gaps,  while  preserving 
benefits  offered  by  the  protest  system  to  promote  competition  and  ensure  the 
integrity,  transparency,  and  accountability  of  the  procurement  process.  An  important 
contribution  of  this  study  is  to  add  an  economics  dimension  to  a  process  mostly 
thought  of  in  legal  terms. 

In  economics,  there  is  no  such  thing  as  a  “free  lunch.”  Operating  a  protest 
system  is  costly.  The  well-documented  advantages  of  allowing  firms  to  protest 
public  procurements  also  introduce  significant  opportunity  costs  and 
transaction  costs.  For  instance,  the  Congressional  Research  Service  (CRS) 
reports  that  “many  ...  acquisition  professionals  are  concerned  that  bid 
protests  can  delay  contract  awards  ...  costing  millions  of  dollars  [and] 
preventing  government  from  getting  the  goods  and  services  it  needs  when  it 
needs  them”  (Schwartz  &  Manuel,  2009,  p.  8). 

Coined  by  latest  recent  Nobel  Prize  winner  in  economics,  Dr.  Oliver 
Williamson,  the  term  transaction  costs  includes  any  non-production-related 


inexpensive  alternative  to  formal  judicial  proceedings.  Today  the  vast  majority  of  protests  are  filed 

with  the  GAO,  and  that  is  the  principal  focus  of  this  study.  Over  a  five-year  period  from  FY2003-2007, 
6,931  cases  were  filed  with  the  GAO — an  average  of  1 ,386  per  year — while  only  328  were  filed  with 

the  Court  of  Federal  Claims  (CFC) — an  average  of  66  per  year  (GAO  and  CFC  statistics  from 
Schaengold,  Guiffre,  &  Gill,  2009,  p.  255). 

4  For  example,  in  GTSI  Corp.  v.  Wildflower  International,  Inc.,  Case  No.  1:09cv123  (JCC), 
memorandum  op.  at  15,  (E.D.Va.  Jul.  17,  2009),  a  Federal  district  court  held  that  a  valid  conspiracy 
claim  may  exist  where  a  contractor  allegedly  tried  to  “punish”  a  competitor  by  organizing  a  campaign 
to  file  “baseless”  bid  protests. 

5  “In  FY2008,  a  single  and  politically  charged  protested  procurement  (the  Boeing  Company’s 
challenge  of  a  contract  awarded  by  the  Air  Force  to  Northrop  Grumman  for  a  new  fleet  of  aerial 
refueling  tankers)  generated  unprecedented  interest  in  the  DoD  acquisition  process”  (Joint  Analysis 
Team  [JAT],  2009,  p.1).  The  current  interest  in  reviewing  the  bid  protest  process  is  largely  due  to 
several  high-value,  high-visibility  protests  recently  sustained  by  the  GAO  (e.g.,  the  combined  value  of 
the  contracts  for  the  contested  aerial  refueling  tanker  [KC-X]  and  Combat  Search  &  Rescue 
Replacement  Helicopter  [CSAR-X]  is  approximately  $65  billion).  There  is  an  extensive  body  of 
literature  that  discusses  the  KC-X  and  other  critical  programs  that  have  been  subjected  to  protests 
(for  example,  see  O’Rourke,  2009). 
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contracting  costs  such  as  the  costs  of  negotiating,  monitoring,  and  managing 
transactions,  and  the  risk  of  opportunism  (see  Melese  et  al.,  2007,  and  Williamson 
1971,  1985,  1979,  1999).  Whereas  a  protest  process  offers  the  benefits  of  an 
oversight  mechanism,  it  also  involves  significant  opportunity  costs  and  transaction 
costs.  For  example,  in  an  August  2007  memo,  then-Acting  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics  John  Young  wrote, 

Protests  are  extremely  detrimental  to  the  warfighter  and  the  taxpayer.  These 
protest  actions  consume  vast  amounts  of  time  [i.e.,  opportunity  costs]  of 
acquisition,  legal,  and  requirements  team  members;  [and]  delay  program 
initiation  and  the  delivery  of  capability  [i.e.,  transaction  costs],  (Schwartz  & 
Manuel,  2009,  p.  8) 

The  economics  approach  would  encourage  senior  leadership  to  identify  and 
evaluate  alternatives  to  protests  that  might  achieve  similar  benefits  at  lower  costs. 

Adopting  an  economics  perspective  would  reveal  two  other  crucial  insights — 
each  an  unintended  consequence  that  further  increases  the  costs  of  a  protest 
system.  First,  protests  can  be  exploited  by  firms  as  a  strategic  bargaining  tool, 
resulting  in  situations  in  which  private  gains  defeat  the  public  interest.  Second, 
governments  can  overreact  (with  a  consequent  risk  of  ballooning  transaction  costs 
from  over-deterrence)  to  avoid  protests.6  The  government’s  goal  of  fighting  agency 
errors  and  corruption  and  promoting  competition  can  be  done  in  other  ways.  By 
recognizing  the  potential  costs  as  well  as  benefits  of  a  protest  system,  the 
economics  approach  considers  alternative  governance  mechanisms  (internal  audits, 
external  audits,  independent  investigations,  GAO  advisory  opinions  and  technical 
assistance  under  title  31  section  526  and  717,  ADR,  investments  in  training  and 


6  Government  agencies  can  overreact  by  tacitly  condoning  the  protest  in  various  ways,  from  failure  to 
manage  the  protest  process  in  a  businesslike  fashion  through  dispute  resolution  flexibilities, 
alternative  dispute  resolution  (ADR),  and  all  available  legal  defenses,  or  from  litigation  strategies 
aimed  at  “buying”  a  GAO  or  judicial  validation  of  agency  decisions  at  the  expense  of  the  program.  In 
the  alternative,  government  agencies  may  be  trying  to  excessively  protest-proof  their  procurement  to 
such  an  extent  that  protest-proofing  displaces  obtaining  value  as  a  procurement  goal. 
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incentives,  greater  use  of  Executive  Order  No.  12979  ADR  measures,  etc.)  that  can 
substitute  for  (or  complement)  protests  and  improve  procurement  outcomes.7 

Acknowledging  the  significant  costs  of  operating  a  protest  system  to  govern 
the  procurement  process  and  encourage  competition  yields  one  of  the  key  insights 
of  this  study:  There  exists  an  optimum  rate  (and  type)  of  protests.  In  combination 
with  a  portfolio  of  other  initiatives  (audits,  investigations,  etc.),  the  optimum  rate  of 
protests  offers  the  most  efficient  and  effective  way  to  ensure  the  integrity  of  the 
acquisition  process,  while  guaranteeing  the  best  possible  procurement  outcomes  for 
our  troops  and  taxpayers.8 

While  it  is  not  easy  to  locate  this  optimum,  the  economics  approach  attempts 
to  identify  and  evaluate  policies  that  could  move  us  closer  to  the  optimum.  One 
important  implication  is  that  a  simple,  often-stated  objective  of  contracting  officials, 
such  as  “reducing  the  rate  of  successful  protests,”  is  not  always  appropriate.9  Two 
additional  factors  need  to  be  considered.  The  first  is  to  identify  where  the  department 
is  relative  to  the  optimum.  The  second  involves  a  careful  consideration  of  how  the 
department  might  choose  to  reduce  and  manage  protests.10 


7  For  example,  a  comparison  of  protests  and  audits  suggests  that  “protests  are  a  more  powerful 
oversight  mechanism,  but  that  they  have  more  adverse  side  effects.  Audits  are  apt  to  result  in  the 
review  of  more  properly  conducted  procurements.  But  protests  result  in  fedmail,  buy-offs  and 
overdeterrence.  ...  [The]  supervision  of  protest  settlement  reduces  harm  from  buy-offs  and  fedmail. 
Similarly,  the  poor  selection  of  cases  by  audit  could  be  improved  by  making  the  auditor  responsive  to 
complaints  by  bidders.  [I]t  may  be  appropriate  to  use  audits  and  protest  simultaneously”  (MMR,  1991, 
pp.  10-11).  Audits,  however,  involve  substantial  investments  in  qualified  workforce  (e.g.,  the  staffs  of 
the  Defense  Contract  Audit  Agency,  the  Offices  of  Inspector  General,  or  congressional  oversight 
committees)  because  the  auditors  must  be  able  to  survey,  detect,  and  resolve  a  pool  of  contracts  that 
may  be  greater  in  scope  than  the  number  of  cases  protested. 

8  Roughly  speaking,  the  optimum  is  when  the  incremental  social  costs  (in  terms  of  extra  delays  and 
cost  overruns)  of  additional  protests  outweigh  the  social  benefits  of  those  protests  (to  deter  corruption 
and  promote  competition,  ensuring  procurement  process  integrity). 


9  For  example,  a  recent  Congressional  Research  Service  (CRS)  report  states,  “This  report  contains 
options  for  Congress  related  to  minimizing  the  number  of  protests”  (Schwartz  &  Manuel,  2009,  p.  1). 

10  Much  of  the  protest  literature  suggests  that  limiting  protests  would  threaten  competition,  thus 
ultimately  undermining  procurement  outcomes.  If  this  is  the  case,  then  restricting  the  rate  of 
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If  the  Department  of  Defense  finds  itself  above  the  optimum,  then  extra 
opportunity  costs  and  transaction  costs  associated  with  protests  tend  to 
overwhelm  the  procurement  process — i.e. ,  by  increasing  delays  and 
contributing  to  cost  overruns  and  performance  gaps.  This  is  reflected  in  a 
recent  statement  that  “The  delay  of  contract  award  . . .  triggered  by  a  GAO 
protest  ...  has  ...  prompted  concerns  about  the  potential  impact  of  protests 
upon  government  agency  operations,  especially  in  the  Department  of 
Defense  (DoD). ’’Schwartz  &  Manuel,  2009,  p.  1). 

If  a  procuring  activity  is  above  the  optimum  rate  of  protests,  then  reducing  the 
rate  of  protests  is  an  appropriate  objective  so  the  burden  does  not  outweigh  the 
benefits.  This  can  be  accomplished  in  two  ways:  by  reducing  the  expected  benefits 
of  a  protest  to  a  “disappointed  bidder”  (including  by  creating  alternative  mechanisms 
to  address  protester’s  concerns)  or  by  increasing  the  expected  costs.* 11 

In  contrast,  if  the  procuring  activity  is  below  the  optimum  rate,  then  the  overall 
benefits  of  the  protests  outweigh  the  extra  (opportunity  and  transaction)  costs.  In  this 
range,  while  reducing  the  rate  of  protests  might  appear  to  have  positive  short-run 
impacts  for  the  activity,  it  could  be  at  the  expense  of  greater  long-term  benefits,  such 
as  preserving  the  integrity  of  the  public  procurement  system  and  encouraging  future 
competition.  Remarkably,  this  study  reveals  two  important  cases  that  demonstrate 


successful  protests  might  reduce  the  direct  costs  of  protests  to  the  procuring  agency,  but  it  would 
raise  new  indirect  costs  from  reduced  competition.  In  contrast  to  the  existing  literature,  this  study 
suggests  that  the  net  effect  on  competition  of  limiting  protests  is  an  empirical  question.  This  raises  the 
possibility  that  carefully  limiting  certain  kinds  of  protests  could  actually  increase  competition. 
Identifying  and  mitigating  conditions  that  trigger  protests  could  actually  increase  competition,  or 
reduce  risk  premiums  that  may  be  implicitly  charged  by  contractors  to  the  government  as  part  of  bids 
or  offers  to  hedge  against  protests. 

11  This  is  a  critical  distinction  emphasized  in  this  study.  The  representative  bidder  model  developed  in 
Section  III  offers  an  illustration.  It  highlights  the  key  differences  in  front  of  a  prospective  bidder  (or 
offeror)  who  is  faced  with  policies  that  limit  protests  either  by  reducing  the  expected  benefits  of  a 
protest  or  by  increasing  the  expected  costs.  A  related  problem  is  when  the  agency  does  not  resolve 
the  protest  in  a  way  that  is  optimal  for  the  particular  procurement  program  at  issue.  For  instance,  if 
an  agency  needlessly  consents  to  a  stay  of  the  procurement  and  then  takes  excessive  corrective 
action  by  cancelling  a  competition  in  its  entirety  instead  of  making  minor  changes  or  brief  re- 
evaluation  of  a  contract  award,  the  protester  would  receive  inappropriate  benefits  from  the  protest. 
The  government  should  have  a  strong  interest  in  reducing  such  windfall  benefits  to  potential 
protesters. 
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that  even  in  this  range,  it  is  possible  to  reduce  protests  and  improve  long-term 
outcomes.  The  key  is  to  carefully  consider  how  the  department  reduces  protests.12 

This  study  follows  the  basic  outline  of  strategic  planning.  The  next  section 
(Background)  answers  the  question,  where  are  we?  Section  III  (An  Economic 
Protest  Framework  and  Change  Candidates)  answers  the  question,  where  do  we 
want  to  go?  Finally,  Section  IV  (Summary  and  Policy  Recommendations)  answers 
the  question,  how  do  we  get  there? 

Section  II  consists  of  two  parts:  a  review  of  insights  from  the  existing  protest 
literature,  and  a  brief  review  of  the  protest  process  along  with  alternative  governance 
mechanisms.  Woven  throughout  this  section  is  a  discussion  of  data  and  survey 
results.  Section  III  consists  of  three  parts:  an  economic  framework  (representative 
bidder  model),13  and  two  sets  of  change  candidates  suggested  by  the  model  to 


12  In  general,  if  the  procuring  agency  finds  itself  below  the  optimum,  more  protests  are  warranted 
since  added  oversight  contributes  to  the  integrity  of  the  procurement  process  and  encourages 
competition.  In  fact,  much  of  the  existing  legal  and  regulatory  literature  implicitly  assumes  procuring 
agencies  are  below  the  optimum.  This  makes  it  appear  somewhat  counterintuitive  that  reducing 
protests  could  actually  increase  competition  since  the  literature  focuses  on  how  reducing  protests 
creates  the  risk  of  reducing  the  participation  of  losing  bidders.  However,  as  discussed  in  Section  II, 
the  trick  is  to  reduce  protests  in  such  a  way  that  it  encourages  sufficient  additional  competition  by 
reducing  the  burden  on  prospective  winning  bidders  of  defending  their  awards  in  order  to  more  than 
offset  the  loss  of  participation  of  prospective  losing  bidders. 

13  A  two-player  game  model  is  being  developed  by  members  of  the  NPS  Protest  Project  team  that 
allows  for  shared  awards  (split  buys).  Split-buy  remedies  are  consistent  with  statutes  restricting 
bundling  or  consolidation  of  government  contracts,  such  as  the  Competition  in  Contracting  Act,  10 
U.S.C.  §  2304;  the  Small  Business  Act,  15  U.S.C.  §  631,  et.  seq.;  and  the  Consolidation  of  Contract 
Requirements  law,  10  U.S.C.  §  2382.  They  are  also  consistent  with  defense  industrial  base  statutes, 
such  as  10  U.S.C.§§  2501-2507.  Split  buys  must  be  carefully  structured  if  the  goal  is  to  reduce  bid 
protests  while  simultaneously  promoting  competition  and  encouraging  efficiency  and  operational 
effectiveness  for  our  troops  and  taxpayers.  The  proposal  is  for  the  contract  split  to  depend  on  the 
relative  value  of  the  contract  offers,  with  the  winner  receiving  a  larger  split  (i.e.,  share  of  the 
procurement)  the  greater  the  difference  between  the  highest  and  second  highest  valued  contractor.  If 
the  second-best  offer  is  close  to  the  best  offer,  then  the  contract  split  would  be  close  to  50:50.  If 
structured  correctly,  such  a  split  strategy  has  the  potential  to  retain  the  benefits  of  competition  by 
encouraging  contractors  to  submit  proposals  that  reflect  what  they  view  as  the  best  government 
value.  Contract  shares  could  be  adjusted  over  time  to  reward  contract  performance  and  cost  savings 
from  process  and  product  innovation.  Finally,  split  buys  increase  competition  for  follow-on  contracts 
since  they  preserve  multiple  procurement  sources.  This  approach  to  protests  deserves  more  careful 
modeling  and  exploration. 


NAVAL  POSTGRADUATE  SCHOOL 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 


-7- 


reduce  the  risk  of  protests — factors  that  reduce  the  expected  benefits  to  a  losing 
bidder  of  a  protest,  and  factors  that  raise  the  expected  costs.  Section  IV  summarizes 
those  factors  (or  change  candidates)  and  suggests  approaches  to  improve  the 
acquisition  process  and  procurement  outcomes  with  an  application  to  the  KC-X 
aerial  refueling  tanker  competition.  The  study  concludes  with  a  list  of  policy 
recommendations  and  recommendations  for  future  research. 
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Background  and  New  Insights 


Most  of  the  legal  and  regulatory  literature  on  bid  protests  focuses  on  the 
benefits  of  a  protest  system.14  Much  less  is  written  about  the  costs.15  This  section 
takes  a  more  balanced  view. 

Bid  protest  systems  are  designed  as  decentralized,  vendor-driven  governance 
mechanisms  to  improve  public-sector  contracting  outcomes.16  Because  a  significant 
fraction  of  public  expenditures  goes  to  acquiring  goods  and  services,17  the  theory  is 
that  allowing  vendors  to  challenge  public  procurements  offers  relatively  inexpensive 
oversight  of  the  procurement  process,  promoting  integrity,  transparency,  and 
accountability  and  encouraging  competition.  A  popular  argument  is  that  a  protest 
system  spurs  competition  by  reducing  entry  barriers  that  arise  from  a  perception  of 
government  biases,  errors,  and  favoritism,  thereby  improving  procurement 
outcomes.  This  argument  appears  in  a  recent  report  to  Congress  by  the  Government 
Accountability  Office  (GAO):  “The  availability  of  a  strong  bid  protest  mechanism 
promotes  competition  in  the  procurement  system  by  providing  contractors  a 


14  The  opening  quotation  of  the  Introduction  offers  an  excellent  example.  The  primary  concerns 
expressed  in  the  literature  are  to  ensure  the  integrity  (probity)  and  equity  of  the  public  procurement 
process.  The  former  “involve[s]  two  aspects:  first,  preventing  actual  corruption — such  as  bribery,  or 
the  award  of  contracts  based  on  personal  interest — and  second,  securing  the  appearance  of  probity 
...  ensuring  that  suppliers  and  the  general  public  have  confidence  in  the  procurement  process” 
(Arrowsmith  et  al.,  2000,  p.  32).  Equity  refers  to  the  equal  treatment  of  providers  to  promote 
competition  and  ensure  that  vendors  are  not  arbitrarily  excluded  from  government  business. 

15  Two  important  exceptions  are  Marshall,  Meurer,  and  Richard  (1991)  and  Metzger  and  Lyons 
(2007). 

16  Another  goal  of  allowing  protests,  expressed  by  the  World  Trade  Organization’s  (WTO)  Agreement 
on  Government  Procurement  and  by  various  Memoranda  of  Understanding  between  the  Department 
of  Defense  and  foreign  governments,  concerns  reciprocal  access  to  each  country’s  defense  contracts 
in  order  to  minimize  discrimination  against  foreign  suppliers  in  domestic  government  purchases.  This 
goal  is  a  legally  binding  obligation  of  the  U.S.  government  that  may  not  be  circumvented.  Yet  another 
goal  of  the  protest  system  is  to  fulfill  constitutional  rights  guaranteed  under  the  First  Amendment’s 
redress-of-grievances  clause  and  the  Fifth  Amendment’s  due-process  clause. 

17  “According  to  USAspending.gov,  between  FY2001  and  FY2008,  the  value  of  contract  actions 
signed  by  DoD  increased  120%”  (Arrowsmith  et  al.,  2000,  p.13). 
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measure  of  confidence  that  concerns  regarding  potentially  unfair  treatment  may  be 
addressed  in  a  neutral  forum”  (Kepplinger,  2009b,  p.  14). 

In  general,  a  protest  can  relate  to  either  the  conduct  of  a  procurement  prior  to 
selecting  a  winner  (i.e. ,  in  the  acquisition  strategy  or  the  solicitation:  Invitation  for  Bid 
[IFB],  Request  for  Proposal  [RFP],  or  Request  for  Quotation  [RFQ]),18  determining 
the  competitive  range,  or  selecting  the  winner.  Pre-award  protests  typically  argue 
that  some  aspect  of  the  solicitation  unfairly  disadvantaged  the  protesting  company. 
Post-award  protests  generally  argue  that  a  procurement  activity  or  official  acted 
improperly  and  that  this  prejudiced  the  losing  protesting  bidder  or  offeror.19  Post¬ 
award  protests  are  the  primary  focus  of  this  study  (Gordon,  2006,  p.  2). 


18  An  agency  generally  develops  a  solicitation  that  identifies  what  the  agency  wants  to  buy,  advertises 
the  solicitation,  identifies  the  method  for  evaluating  offers,  and  sets  a  deadline  for  the  submission  of 
bids  or  proposals.  The  agency  then  evaluates  the  bids  or  proposals  based  on  the  criteria  set  forth  in 
the  solicitation  and  awards  the  contract  to  the  winning  firm  (Schwartz  &  Manuel,  2009,  p.  2).  An  IFB 
begins  with  an  announcement  of  mandatory  specifications.  The  award  is  then  made  to  the  vendor 
that  satisfies  the  specifications  and  offers  the  lowest  price  in  a  sealed-bid  competition.  The  RFP  lists 
desirable  specifications  and  assigns  points  to  offers  that  contain  those  features.  A  winning  bid  is  often 
determined  by  summing  the  points  for  technical  merit  and  then  choosing  the  proposal  that  offers  the 
best  value  in  terms  of  cost  and  performance/schedule.  The  scoring  function  assigns  weights  to  cost 
and  different  quality  attributes.  Unlike  an  IFB,  a  proposal  submitted  by  a  firm  is  not  necessarily  its  final 
offer.  Often,  the  agency  conducts  written  or  oral  negotiations  with  all  responsible  offerors  within  a 
competitive  range  determined  by  the  Procurement  Official  (PO).  (For  the  purposes  of  this  paper,  the 
term  Procurement  Official  encompasses  Contracting  Officers,  Senior  Procurement  Executives,  and 
others  with  procurement  authority).  During  this  time,  the  PO  gives  offerors  an  opportunity  to  correct 
deficiencies  and  clarify  ambiguities.  Firms  in  the  competitive  range  revise  their  offers  and  submit 
“Best  and  Final  Offers.”  POs  exercise  substantial  discretion  in  determining  specifications  and 
evaluation  criteria.  (This  study  will  often  use  the  terms  bid  and  offer  interchangeably.) 

19  The  issue  of  prejudice  is  central  once  a  protester  convinces  a  forum  that  the  agency  acted 
improperly.  Prejudice  refers  to  the  harm  the  protester  suffered  from  the  agency’s  improper  action.  If  a 
substantial  weight  is  placed  on  prejudice,  then  even  if  an  agency  acted  improperly,  a  protest  can  still 
be  denied  if  the  improper  actions  did  not  prejudice  the  protesting  vendor,  i.e.,  had  no  impact  on  their 
chances  to  win  the  competition  (Gordon,  2006,  p.  11).  This  would  be  an  example  of  a  meritorious 
protest  denied  for  lack  of  prejudice.  By  sustaining  protests  only  if  a  protester  was  prejudiced  (i.e., 
could  have  won  the  competition),  the  tribunal  is  able  to  ensure  that  agencies  are  not  told  to  redo 
procurements  based  on  mere  technicalities.  The  alternative  is  for  a  review  forum  like  the  GAO  to 
leave  agencies  complaining  that  protests  are  sustained  on  technicalities  that  would  have  had  no 
bearing  on  the  selection  of  a  contractor.  Whereas  hearing  protests  from  those  who  were  not 
prejudiced  might  shine  the  light  on  parts  of  the  procurement  system  that  need  improvement,  doing 
so  might  also  significantly  increase  transaction  costs.  In  this  case,  procurement  officials  would  be 
forced  to  practice  a  form  of  “defensive  medicine,”  using  up  valuable  resources  to  avoid  even  the  hint 
of  a  protest.  This  also  invites  the  possibility  of  prospective  bidders  acting  strategically  and  using  the 
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The  literature  generally  focuses  on  two  players:  a  disappointed  bidder20  and 
the  government.  In  fact,  the  government  can  be  broken  into  several  distinct  players, 
each  of  which  might  expect  a  challenge  from  a  disappointed  bidder:  procurement 
officials  (POs)  and  their  department  or  activity,  the  quasi-judicial  Government 
Accountability  Office  (GAO),  and  the  Court  of  Federal  Claims  (COFC). 

Other  key  stakeholders  occasionally  overlooked  in  the  protest  literature 
include  other  bidders  (especially  the  “winning  bidder,”  eager  to  defend  the  award), 
our  troops  (who  ultimately  stand  to  benefit  from  the  procurement),  and  taxpayers 
(who  foot  the  bill).21  The  aim  of  this  study  is  to  represent  the  best  interests  of  the  last 
two  players:  our  troops  and  the  taxpayers.  The  report  investigates  the  protest 
process  with  the  aim  of  identifying  opportunities  to  enhance  the  efficiency  and 
effectiveness  of  government  procurements  in  order  to  obtain  the  greatest  (troop) 
value  for  (taxpayer)  money.22 


threat  of  a  protest  as  a  bargaining  tool.  These  two  concerns  are  discussed  in  more  detail  in  Chapters 
II  A  and  B  .  In  contrast,  audits  as  accountability  measures  are  focused  primarily  on  compliance  and 
do  not  have  an  independent  mechanism  of  measuring  prejudice.  While  audits  may  reduce  costs  for  a 
particular  transaction,  they  can  also  increase  costs  because  of  this  lack  of  focus  on  prejudice. 

20  Other  terms  besides  disappointed  bidder  found  in  the  literature  include  disappointed  offeror, 
unsuccessful  offeror,  excluded  offeror,  and  interested  party.  For  the  purposes  of  this  study,  these 
terms  are  used  interchangeably  and  basically  refer  to  a  company  that  has  standing,  or  is  allowed  to 
protest  the  solicitation  or  award  of  a  contract.  “Interested  party  ...  means  an  actual  or  prospective 
offeror  whose  direct  economic  interest  would  be  affected  by  the  award  of  a  contract”  (FAR,  Subpart 
33.1 01 — Definitions). 

21  In  an  article  entitled  “Constructing  a  Bid  Protest  Process:  Choices  Every  Procurement  Challenge 
System  Must  Make,”  Daniel  Gordon  (2006)  focuses  on  four  principal  parties:  the  disappointed  offeror 
who  is  denied  a  contract  award  or  the  potential  offeror  who  is  excluded  from  competition,  the 
acquiring  agency,  the  public  at  large  and  their  elected  representatives,  and  an  intervening  offeror  or 
successful  awardee.  Each  has  a  different  objective  in  resolving  the  protest.  The  unsuccessful  offeror 
seeks  a  forum  to  air  complaints,  to  learn  as  much  information  as  possible  about  the  denial  or 
exclusion  of  their  offer,  and,  ultimately,  to  obtain  some  type  of  meaningful  relief.  The  acquiring 
agency  seeks  to  resolve  the  protest  in  a  way  that  minimizes  the  impact  on  the  efficiency  and 
effectiveness  of  the  acquisition  process.  The  public  seeks  a  resolution  that  promotes  the  integrity, 
transparency,  and  accountability  of  the  acquisition  system.  The  successful  awardee  (or  intervening 
offeror)  seeks  a  resolution  that  supports  the  original  award  (Gordon,  2006,  p.  4). 

22  “The  federal  procurement  system  was  designed  by  Congress  to  leverage  maximum  public  benefit 
from  scarce  taxpayer  funds  through  three  guiding  principles:  competition,  integrity,  and  transparency. 
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This  section  consists  of  two  parts:  a  review  of  insights  from  (and  gaps  in)  the 
literature,  and  a  brief  description  of  the  protest  process.  Woven  throughout  this 
section  is  a  discussion  of  data,  legal  authorities,  the  protest  process,  the  literature, 
and  some  survey  results.23 

A.  A  Review  of  Insights  from  the  Literature 

Whereas  the  most  common  term,  and  the  term  used  in  this  study,  is  bid 
protest,  the  United  Nations  Commission  on  International  Trade  Law 
(UNCITRAL)  refers  to  reviews,  while  the  World  Trade  Organization’s  (WTO) 
Agreement  on  Government  Procurement  uses  the  term  challenges  (see 
generally,  Gordon,  2006).  As  stated  in  the  U.N.  Commission  on  International 
Trade  Law,  UNCITRAL  Model  Law  on  Procurement  of  Goods,  Construction 
and  Services,  chap  VI,  Guide  to  Enactment,  (1994  p.30)  “A  review  process  ... 
helps  to  make  [acquisition  regulations]  to  an  important  degree  self-policing 
and  self-enforcing,  since  it  provides  an  avenue  for  review  to  ...  contractors, 
who  have  a  natural  interest  in  monitoring  compliance  [of  procurement  law]  by 
procuring  entities”  (UNCITRAL,  1994,  p.  30). 

A  key  feature  of  protests  that  differentiates  them  from  other  methods  of 
policing  government  procurements  (audits,  investigations,  etc.)  is  that  they  serve  as 
a  decentralized  oversight  mechanism.  Disappointed  bidders  serve  as  “private 
attorneys  general”  in  the  sense  that  they  direct  the  government  to  investigate  certain 
procurements  through  their  protests  (see  Marshall  et  al. ,  1991).  This  allows  private- 
sector  vendors  to  decide  where  the  government  spends  its  investigative  and 
oversight  resources.  This  is  in  sharp  contrast  to  traditional,  centralized  mechanisms 
such  as  audits,  inspectors  general,  or  criminal  prosecutions,  in  which  government 
officials  are  the  ones  that  decide  which  procurements  to  review.24 


Bid  protests  play  an  important  role  in  ensuring  integrity  in  the  federal  procurement  system  while  ... 
enhancing  transparency  and  accountability”  (JAT,  2009,  p.  1). 

23  Survey  results  were  compiled  by  Benishek  et  al.  (2011)  and  appear  in  a  companion  report,  Better 
Acquisition  Management  through  Alternative  Dispute  Resolution  (ADR)  and  Other  Best  Practices  for 
Preventing  and  Resolving  Bid  Protests,  published  by  the  Naval  Postgraduate  School  Acquisition 
Research  Program  as  part  of  this  study  effort  on  the  topic  of  bid  protests. 

24  In  terms  of  reducing  fraud  (corruption)  in  the  solicitation  or  selection  process,  whistleblower  reports 
to  the  Inspector  General  or  the  Justice  Department  are  another  viable  de-centralized  substitute  for 
contractor  protests.  Similar  to  qui  tam  actions  that  arise  under  the  False  Claims  Act,  31  U.S.C. 
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Protest  systems  serve  multiple  purposes.  The  federal  government’s  protest 
process  mostly  focuses  on  deterring  and  correcting  fraud  (intentional  violations  of 
procurement  laws  and  regulations)  and  errors  (inadvertent  violations  of  procurement 
laws  and  regulations)  and  on  encouraging  competition.25 

Fundamentally,  bid  protest  systems,  like  audit  systems,  serve  a  procurement 
oversight  function.  They  provide  a  means  of  monitoring  the  activities  of 
government  procurement  officials,  enforcing  compliance  with  procurement 
laws  and  regulations,  and  correcting  incidents  of  improper  government  action. 
...  [Enforcing  compliance  with  procurement  laws  implicates  not  just  high 
standards  of  integrity,  but  also  ...  the  maximization  of  competition.  (Troff, 
2005,  pp.  118,  120) 

Bid  protests  are  one  of  several  different  mechanisms  used  by  federal,  state, 
and  local  governments  and  other  countries  around  the  world  to  increase  competition 
and  reduce  fraud,  waste,  and  abuse  in  government  contracting. 

Bid  protest  procedures  are  now  well-established  as  a  key  feature  of  every 
major  national  procurement  regime  as  well  as  the  international  trade 
agreements  that  address  government  procurement — such  as  the  WTO 
Agreement  on  Government  Procurement,  the  North  American  Free  Trade 
Agreement  (NAFTA),  and  the  European  Union  Procurement  Directives.  (Troff, 
2005,  p.129) 

Protest  systems  are  designed  to  enhance  the  accountability  of  procurement 
officials  and  government  agencies  and  to  improve  bidder  participation.  By  allowing 
protests,  the  government  can  increase  competition  if  this  reduces  barriers  to  entry 
created  by  a  perception  of  favoritism  or  lack  of  integrity  and  transparency  in  the 
procurement  process.  Not  only  does  a  protest  system  serve  as  a  deterrent  to 


section  3729,  potential  whistleblowers  might  be  induced  to  step  forward  if  they  were  offered  the  same 
15-30%  of  funds  recovered/saved  in  the  procurement  as  may  be  recovered  by  private  qui  tarn 
realtors.  The  economics  approach  promoted  in  this  study  suggests  identifying  and  evaluating 
alternative  governance  mechanisms  that  achieve  similar  aims  to  a  protest  system  at  a  lower  cost. 

25  “Bid  protest  systems  ...  are  characterized  by  their  speed  and  efficiency,  the  meaningfulness  and 
independence  of  their  review  processes,  and  their  ability  to  provide  meaningful  relief”  (Troff,  2005,  p. 
116). 
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improper  conduct — improving  the  integrity  and  fairness  of  the  procurement 
process — but  it  can  also  help  clarify  and  shape  procurement  law.26 

However,  there  is  a  dark  side.  Achieving  these  goals  increases  opportunity 
costs  and  transaction  costs  and  can  lead  to  lengthy  and  costly  procurements.27 

A  narrow  goal  for  the  Defense  Department  is  to  minimize  transaction  costs  in 
order  to  promptly  and  efficiently  complete  an  acquisition.  This  argues  for  limiting 
delays  and  disruptions  from  protests  by  reducing  the  expected  benefits  of  a  protest 
to  a  disappointed  bidder,  or  by  increasing  the  expected  costs. 

Proposals  to  accomplish  this  have  included:  narrowing  standing  (eligibility), 
setting  stricter  time  limits  for  filing  protests  (and  for  deciding  them),  encouraging 
alternative  dispute  resolution  (ADR),  raising  filing  fees,  setting  fines  for  frivolous 
protests,  instituting  new  rules  such  as  restricting  the  ability  to  institute  frequent  or 
repeated  protests,  or  establishing  the  loser  pays  method  (for  cases  in  which  a 
protest  has  merit  but  is  ultimately  denied  because  the  dispute  involves  a  mere 
technicality  and  there  is  lack  of  prejudice).28  Although  a  fear  generally  expressed  in 


26  Of  course,  taxpayers’  representatives  in  Congress  often  have  zero  tolerance  in  public  procurement 
for  certain  violations,  such  as  conflicts  of  interest,  kickbacks,  misrepresentation,  and  procurement 
integrity  violations.  For  instance,  under  the  False  Claims  Act,  delivery  of  quality  products  or  services 
is  not  an  allowable  defense,  (see  Longhi  ex  rel.  United  States  v.  Lithium  Power,  575  F.3rd  459  (5th 
Cir.  2009).  However,  it  may  be  appropriate  to  consider  whether  all  violations  currently  redressed 
through  the  protest  process  for  all  types  of  disappointed  bidders  should  continue  to  be  redressed 
through  remedies  that  have  substantial  cost  and  delay  impacts  on  the  acquisition  system 

27  “A  number  of  high-profile  government  acquisitions  have  experienced  extensive  delays  as  a  result  of 
GAO  bid  protest  decisions.  For  example,  on  Jun  18,  2008,  GAO  sustained  Boeing’s  protest  of  the  Air 
Force’s  award  of  a  contract  to  Northrop  Grumman  [for  the  KC-X  tanker  aircraft].  ...  Identifying  which 
protests  tend  to  experience  the  longest  delays — and  working  to  minimize  such  delays — could  help  the 
government  save  hundreds  of  millions  of  dollars  and  receive  the  goods  and  services  it  needs  when  it 
needs  them”  (Schwartz  &  Manuel,  2009). 


28 

Note  that  restricting  the  number  of  protests  is  unconstitutional  on  First  and  Fifth  Amendment 
grounds.  The  frequently  stated  recommendation  to  limit  the  number  of  non-frivolous  protests  would 
violate  the  First  Amendment  right  to  petition  the  government  for  the  redress  of  grievances  and  the 
Fifth  Amendment  right  to  due  process.  As  noted  in  the  companion  report  of  Benishek  et  al.  (2011,  pp. 
110),  Federal  courts  disfavor  broad  limitations  on  access  to  the  legal  process.  For  example,  the 
Supreme  Court  held  in  Bill  Johnson’s  Restaurants,  Inc.  v.  National  Labor  Relations  Board  (461  U.S. 
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the  legal  and  regulatory  literature  is  that  limiting  protests  will  inhibit  competition  and 
result  in  higher  costs,29  those  costs  could  be  more  than  offset  by  reductions  in  delays 
and  by  disruptions  from  fewer  protests. 

In  reviewing  the  literature,  there  appear  to  be  two  underappreciated  risks 
(costs)  associated  with  the  operation  of  the  current  protest  system.  One  is  the 
strategic  use  of  protests  (e.g.,  stays,  injunctions,  or  delays)  by  firms  to  obtain 
competitive  advantage  (or  to  recoup  bid  proposal  costs).30  The  other  is  the  risk  of 


731  (1983)),  that  a  Federal  agency  cannot  halt  lawsuits  brought  even  for  improper  motives  unless 
those  lawsuits  are  based  on  “intentional  falsehoods  or  on  knowingly  frivolous  claims,”  or  otherwise 
lack  a  reasonable  basis.  In  another  case,  California  Motor  Transport  Co.  v.  Trucking  Unlimited  (404 
U.S.  508  (1972)),  the  Supreme  Court  held  that  Federal  antitrust  laws  may  penalize  businesses 
bringing  lawsuits  and  petitions  to  Federal  agencies  only  if  such  petitions  and  lawsuits  are  “a  mere 
sham  to  cover  what  is  actually  nothing  more  than  an  attempt  to  interfere  directly  with  a  business 
relationship  of  a  competitor.”  Federal  appellate  courts  also  identified  two  limited  ways  which  can 
render  a  legal  action  frivolous:  “First,  a  legal  action  is  considered  ‘frivolous  as  filed’  when  a  plaintiff  or 
appellant  grounds  its  case  on  arguments  or  issues  ‘that  are  beyond  the  reasonable  contemplation  of 
fair-minded  people,  and  no  basis  for  [the  party’s  position]  in  law  or  fact  can  be  or  is  even  arguably 
shown.’ .  .  .  Second,  a  legal  action  is  considered  ‘frivolous  as  argued’  when  a  plaintiff  or  appellant  has 
not  dealt  fairly  with  the  court,  has  significantly  misrepresented  the  law  or  facts,  or  has  abused  the 
judicial  process  by  repeatedly  litigating  the  same  issue  in  the  same  court.”  (GAO,  B-401197,  2009, 

1 1 ).  However,  options  include  agency  policies  requiring  mandatory  consideration  of  stay  overrides, 
requiring  vigorous  objections,  setting  stricter  time  limits  for  deciding  or  resolving  protests,  mandating 
alternative  dispute  resolution  (ADR)  as  the  default  resolution  mechanism,  or  other  approaches  such 
as  replicating  sanctions  for  frivolous  protests  available  at  the  Court  of  Federal  Claims  in  GAO 
protests,  or  instituting  rules  such  as  the  posting  of  bonds  for  the  expenses  of  delays  resulting  from 
stays  of  protests  that  are  ultimately  not  sustained.  In  addition,  the  standard  of  review  at  the  GAO  may 
be  adjusted  from  the  more  relaxed  and  subjective  “reasonableness”  standard  to  the  “arbitrary, 
capricious,  abuse  of  discretion,  or  otherwise  not  in  accordance  with  law”  standard  used  by  the  COFC 
under  the  Administrative  Procedures  Act  (Choice  of  Forum  for  Federal  Government  Contracts  Bid 
Protests,  at  298  (2009).  Further,  agencies  can  be  encouraged  not  to  allow  post-award  bid  protests 
challenging  the  evaluation  and  the  conduct  of  source  selection  to  result  as  a  matter  of  course  in  pre¬ 
award  corrective  actions,  such  as  total  cancellation  of  solicitation  and  full  re-competition. 

29  “Attempts  to  disincentivize  protests  ...  may  have,  on  balance,  the  unintended  consequence  of 
harming  the  federal  procurement  system  by  discouraging  participation  in  federal  contracting  and,  in 
turn,  limiting  competition”  (Kepplinger,  2009,  p.  12). 

30  Another  strategic  use  of  protests  is  so-called  “fishing  expeditions.”  These  are  attempts  to  obtain 
valuable  information  about  competitors,  selection  criteria,  etc.,  to  use  in  current  or  future 
competitions.  Although  issuance  of  protective  orders  are  meant  to  prevent  firms  from  accessing 
information  from  an  agency  or  competitor  that  might  be  used  to  unfair  advantage,  it  is  plausible  that 
information  gleaned  in  the  protest  process  could  turn  out  to  have  significant  strategic  value.  As 
discussed  throughout  the  companion  report,  Benishek  et  al.  (2011),  better  quality  debriefings  can  play 
a  significant  role  in  reducing  or  mitigating  protests. 
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over-deterrence,  or  a  proliferation  of  defensive  actions  by  procurement  officials 
(POs)  to  avoid  protests.31 

1.  The  Strategic  Use  of  Protests  by  Firms 

Prospective  bidders  can  use  the  (implicit  or  explicit)  threat  of  filing  a  protest 
as  a  strategic  bargaining  tool  in  negotiations.32  This  is  a  uniquely  credible  threat  due 
to  the  significant  delays  that  arise  in  disputed  procurements.33  In  fact,  filing  a  protest 
with  the  Government  Accountability  Office  (GAO) — the  most  popular  forum — 
guarantees  an  automatic  stay  of  up  to  100  days.34 

In  order  to  avoid  the  threat  of  costly  delays,  procurement  officials  are 
understandably  tempted  to  engage  in  preemptive  settlements.  Though  a  large 
number  of  disputes  in  our  legal  system  are  settled  rather  than  litigated  (Marshall  et 


31  A  risk  is  that  the  government’s  focus  (i.e. ,  the  pendulum)  would  shift  from  the  execution  of  contracts 
to  an  over-emphasis  on  compliance,  potentially  harming  acquisition  outcomes.  Another  risk, 
addressed  in  detail  in  the  companion  report,  is  that  agencies,  in  effect,  may  be  tacitly  condoning  filing 
and  litigation  of  bid  protests  in  order  to  purchase  a  seal  of  approval  or  obtain  authoritative  direction 
from  the  GAO  (see  Benishek  et  al. ,  201 1 ,  p.  69). 


32  The  concern  over  frivolous  protests  may  also  be  relevant  for  tactical  reasons  as  a  protestor 
strategy  to  reveal  key  information  for  current  or  future  competitions.  The  risk  of  abuse  could  motivate 
raising  filing  fees,  punishing  vendors  that  file  frivolous  protests,  or  compelling  unsuccessful  protestors 
to  reimburse  the  government  (and  awardee)  for  the  cost  of  defending  against  the  protests.  The 
danger  is  that  disappointed  bidders  with  legitimate  issues  might  be  deterred,  fearing  the  contracting 
agency  might  retaliate  in  future  competitions.  These  concerns  could  be  examined  by  setting  up 
Statistical  Hypothesis  tests  and  exploring  Type  I  and  Type  II  errors. 

33  In  the  companion  report  presenting  a  recent  NPS  survey  of  acquisition  officials,  70%  (14/20 
participants)  responded  that  strategic  behavior  by  disappointed  offerors  was  at  least  a  moderate 
aspect  that  precluded  the  effective  avoidance  or  resolution  of  bid  protests  (see  Benishek  et  al.,  201 1 , 
pp. 71-72  ).  The  GAO  is  aware  of  the  risk  of  delays:  “Despite  the  variability  in  protest  filings  from  one 
year  to  the  next,  our  Office  minimizes  the  delay  and  disruption  associated  with  protest  filings  by 
resolving  protests  as  expeditiously  as  practicable.  In  fact,  during  the  last  5  years,  we  have 
consistently  closed  more  than  half  of  all  DOD  protests  within  30  days  of  when  they  were  filed” 
(Kepplinger,  2009b,  p.  10). 

34  “The  filing  of  a  GAO  protest ...  triggers  an  automatic  stay  of  contract  award  or  performance  that  can 
interrupt  agencies’  procurement  initiatives  for  as  long  as  the  protest  is  pending”  (Schwartz  &  Manuel, 
2009,  p.  3).  Note  that  the  alternative  bid  protest  forum,  the  Court  of  Federal  Claims  (CFC),  has  much 
stricter  requirements  for  a  disappointed  bidder  to  obtain  an  injunction. 
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al. ,  1991), 35  so-called  Fed-mail  and  Buy-off  settlements  made  to  avoid  protests  can 
be  both  subtle  and  costly.36  They  include  anything  from  subtly  restructuring 
solicitations  in  order  to  discriminate  for  or  against  certain  vendors,  to  guaranteeing 
future  work  or  subcontracts,  to  the  payment  of  “consulting”  fees  or  other  forms  of 
cash  payments. 

Bid  protest  disputes  often  give  rise  to  significant  operational  delays. 
Sometimes,  rather  than  litigate  the  bid  protest,  and  then  correct  the  flaws  in 
its  procurement,  an  activity  will  try  to  “buy  off”  a  bid  protester  with  a  monetary 
settlement.  This  practice  is  known  as  “Fedmail”.37  (Office  of  the  General 
Counsel,  2004,  p.  4-83) 

Roughly  speaking,  it  is  useful  to  think  of  a  Fed-mail  settlement  as  a  response 
by  procurement  officials  to  threats  by  a  prospective  bidder  to  impose  a  protest-driven 
delay  of  an  acquisition — primarily  based  on  technicalities  that  violate  the  letter  of  the 
law.  A  Buy-off  settlement  is  similar  except  that  it  is  initiated  by  the  procurement 
official  to  avoid  the  penalty  of  a  protest-driven  delay  due  to  an  (intentional  or 
unintentional)  error  or  omission  in  the  procurement  process  likely  to  violate  the  spirit 
of  the  law.38 


35  A  “survey  by  the  American  Bar  Association  (1989)  found  that  half  of  all  federal  agencies  had  settled 
protests  to  “simply  move  forward  with  the  procurement.  ...  POs  [procurement  officials]  often  settle  by 
enhancing  the  terms  of  other  contracts  that  the  protester  currently  has  with  the  procuring  agency” 
(Marshall  et  al.,  1994,  p.  300). 

36  Procurement  officials  can  subvert  the  protest  system  by  paying  protestors  through  Fed-mail  or  Buy- 
offs  to  abandon  challenges  to  procurement  decisions.  These  could  include  subcontracts,  future 
contracts,  split  buys,  cash  settlements,  etc. 

37  In  ADP  Bid  Protests:  Better  Disclosure  and  Accountability  of  Settlements  Needed  (Government 
Accountability  Office  [GAO],  1990,  p.  31),  the  GAO  indicated  that  it  would  question  the  propriety  of 
Fed-mail  payments,  if  and  when  it  came  across  them. 

38  Such  cash  or  in-kind  settlements  produce  a  negative  externality  for  taxpayers  that  can  defeat  the 
positive  private-attorney-general  role  of  protesters.  “If  the  agency  problem  is  severe,  the  threat  of 
protest  ...  induces  the  [procurement  official]  (or  Awardee)  to  offer  a  cash  or  in-kind  settlement  to 
protesters.  We  refer  to  [this]  as  ‘buy-off  settlements”  (Marshall  et  al.,  1991,  p.  9).  Traditionally,  Fed¬ 
mail  referred  to  cash  settlements  in  bid  protest  cases — a  practice  Congress  sought  to  discourage  by 
requiring  public  reporting  of  delay-avoiding  cash  settlements  for  protests  of  information  technology 
when  the  GSA  Board  of  Contract  Appeals  had  jurisdiction  over  such  protests.  This  problem  was 
evaluated  by  the  GAO  in  its  March  1990  report,  ADP  Bid  Protests:  Better  Disclosure  and 
Accountability  of  Settlements  Needed,"  GAO/GGD-90-13  (March  1990).  As  discussed  in  the  Senate 
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Ironically,  the  many  ways  in  which  the  procurement  process  can  be 
corrupted — which  bid  protest  systems  are  meant  to  deter  and  correct — can  reappear 
in  well-intentioned,  but  misguided,  Fed-mail  and  Buy-off  settlements  by  agencies 
trying  to  keep  an  acquisition  on  track. 

At  the  contract  formation  phase  ...  evaluation  criteria  in  the  request  for 
proposal  or  tender  documents  could  be  drafted  to  favor  a  particular  supplier 
or  service  provider  or ...  to  emphasize  weaknesses  of  a  particular  competitor. 
Later ...  the  evaluation  criteria  could  be  misapplied  or  otherwise  further 
defined  or  amended  after  proposal  or  tender  receipt.  ...  [A]dvance  information 
could  be  provided  to  a  particular  favored  supplier.  Other  techniques  [might 
include]  failing  to  solicit  proposals  or  tenders  from  the  competitors  of  a 
favored  supplier,  wrongfully  restricting  the  tender  pool,  soliciting  offerors 
known  to  be  inferior  to  a  favored  supplier,  ...  or  rejecting  legitimate  proposals. 
(Troff,  2005,  p.  119,  footnote  22) 


Report  103-258  on  the  Federal  Acquisition  Streamlining  Act  of  1994,  adopted  by  the  Senate 
Governmental  Affairs  Committee  on  August  30,  1994: 

In  1990,  GAO  addressed  the  settlement  process  at  the  Board.  This  review  was  in  response 
to  reports  that  disappointed  offerors  were  filing  protests  of  contract  awards,  and  to  avoid 
procurement  delays,  agencies  would  pay  protesters  to  drop  their  claims.  It  also  followed 
GAO's  study  of  the  Census  Bureau's  handling  of  a  protests  of  a  multimillion  dollars 
procurement  of  ADP  for  then-upcoming  census  in  which  $1.1  million  was  paid  by  the  agency 
to  settle  the  claims  without  a  review  of  the  merits. 

GAO  found  that  nearly  half  the  protests  it  sampled  were  settled  before  a  decision  was  issued 
by  the  Board.  Slightly  more  than  half  of  that  number  involved  cases  where  the  terms  of  the 
settlement  were  disclosed.  Although  GAO's  sample  of  cases  did  not  show  a  high  incidence  of 
money  changing  hands,  GAO  noted  that  such  settlements  should  be  an  issue  of  concern. 
Indeed,  GAO  concluded:  [l]f  an  agency  offers  monetary  settlements  solely  to  avoid 
operational  delays  resulting  from  CICA's  suspension  procedures,  we  believe  there  is  no  basis 
for  the  settlement.  Further,  we  would  question  the  appropriateness  of  monetary  settlements 
where  the  agency  (1)  thought  the  protest  had  no  merit  or  (2)  chose  not  to  correct 
procurement  flaws  that  could  be  corrected,  but  settled  with  money  because  it  would  take  less 
time.  ***  Because  not  all  settlements  are  disclosed,  the  fact  that  our  review  did  not  show  a 
high  incidence  of  Fedmail  is  not  conclusive  evidence  that  there  is  no  problem.  ***  GAO 
recommended  that  Congress  act  legislatively  to  require  that  settlement  terms  be  disclosed  to 
the  Board,  and  section  1436  would  address  the  issue. 

FASA  Section  1436,  however,  effectively  disappeared  with  the  demise  of  the  bid  protest  jurisdiction  of 
the  General  Services  Administration  Board  of  Contract  Appeals  (GSBCA).  No  equivalent 
transparency  requirement  exists  today  in  agency,  GAO,  or  COFC  protests.  Congress  should 
consider  rectifying  this  transparency  gap. 
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There  is  a  serious  risk  that  procurement  officials  might  succumb  to  strategic 
threats  of  an  automatic  stay  initiated  by  prospective  bidders/offerors  attempting  to 
improve  their  bargaining  position. 

Companies  may  ‘game’  the  process,  using  the  stay  to  exact  concessions  from 
agencies  and  to  disrupt  the  ...  business  opportunities  of  rivals  who  bested 
them  in  competition.  For  example,  an  incumbent  contractor  that  loses  the 
competition  ...  may  file  a  protest  simply  to  stay  performance  of  the  new 
contract  and  extend  its  current  contract  for  the  duration  of  the  GAO  protest. 

As  long  as  the  marginal  profit  earned  by  extending  the  legacy  contract 
exceeds  the  cost  of  the  protest  ...  the  temptation  to  engage  in  strategic 
behavior  is  always  present.  Alternatively,  a  failed  bidder  may  stay  the  award 
through  a  GAO  protest  and  then  seek  settlement  with  the  awardee  [or 
agency]  by  getting  a  portion  of  the  contract  as  a  subcontractor.  (Metzger  & 
Lyons,  2007,  p.  1240) 

As  emphasized  by  Marshall,  Meurer,  and  Richard  (1991),  protest  forums 
“should  not  grant  standing  to  firms  whose  sole  potential  purpose  is  to  use  the  costs 
and  delays  inherent  in  the  ...  [protest]  process  for  [private]  gain”39  (p.  19). 

The  cost  of  preventing  or  deterring  inappropriate  settlements  is  an  additional 
transaction  cost  not  clearly  understood  or  appreciated  in  the  current  operation  of  the 
protest  process.  A  recommendation  of  this  study  is  that  agencies  and  departments 
evaluate  the  risk  that  protest  systems  might  inadvertently  motivate  inappropriate 
settlements  in  order  to  protect  procurement  actions.40 


39  Exploring  the  additional  possibility  of  settlements  occurring  between  firms,  Marshall  et  al.  (1994) 
conclude,  “An  inter-firm  settlement ...  is  essentially  equivalent ...  to  a  bid-rigging  agreement.  ...  [A] 
payment  by  one  firm  to  another  in  exchange  for  a  decision  not  to  participate  in  a  procurement  would 
certainly  be  subject  to  antitrust  action”  (p.  308).  They  go  on  to  argue,  “Inter-firm  settlements  of 
protests  should  be  banned.  Such  settlements  produce  outcomes  that  are  identical  to  those  that  would 
occur  with  explicit  collusion”  (p.  314).  In  contrast,  one  can  imagine  cases  between  an  agency  and  a 
contractor  in  which  the  social  benefits  of  a  well-crafted  (and  legal)  settlement  (to  avoid  litigation 
expenses,  delays,  extra  transaction  and  opportunity  costs,  etc.)  could  exceed  the  social  costs. 
Regardless,  this  study  recommends  any  significant  settlements  be  subject  to  review. 

40  Another  recommendation  is  to  extend  section  1436  of  the  Federal  Acquisition  Streamlining  Act  of 
1994  to  protests  at  the  GAO,  the  Court  of  Federal  Claims,  and  other  agencies. 
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Especially  troublesome  in  this  context  is  a  protest  measure  developed  and 
routinely  reported  by  the  GAO  called  the  effectiveness  rate  (for  example,  see 
Kepplinger,  2008).  This  measure  captures  “the  percentage  of  protesters  obtaining 
relief — either  through  a  protest  being  sustained,  or  through  voluntary 
action[emphasis  added]  taken  by  the  agency”  (Kepplinger,  2008).  Unfortunately,  in 
the  latter  case,  so-called  “voluntary”  actions  could  easily  consist  of  unproductive 
Fed-mail  or  Buy-off  settlements.41  If  so,  this  would  not  only  undermine  but  also 
directly  contradict  a  recent  conclusion  drawn  by  the  Congressional  Research 
Service  (CRS)  that  “the  effectiveness  rate  may  be  a  good  way  to  measure  the 
number  of  protests  that  have  actual  or  potential  merit”  (Schwartz  &  Manuel,  2009,  p. 
5). 


The  risk  of  Fed-mail  and  Buy-off  settlements  warrants  a  serious  rethinking  by 
the  GAO  of  its  protest  effectiveness  rate  measure.  It  also  cautions  departments, 
agencies,  and  Congress  against  implementing  any  analysis  or  recommendations 
that  utilize  this  measure. 

Another  serious  risk  is  that  agencies  may  tacitly  condone  bid  protests  for 
various  improper  reasons,  such  as  to  obtain  direction  or  a  “seal  of  approval”  from  the 
GAO.  Trends  such  as  increases  in  corrective  actions,  highlighted  by  the 
Congressional  Research  Service  (CRS),  42  and  underutilization  of  legal  tools 


41  Any  increase  in  the  effectiveness  rate  could  indicate  that  government  agencies  are  over-generously 
engaged  in  Fed-Mail  or  Buy-off  settlements  to  keep  procurements  on  schedule  and  minimize  delays, 
or  that  they  are  giving  voluntary  aid  to  protestors  to  avoid  negative  publicity. 

42  When  agencies  over-protect  or  under-defend  bid  protests,  each  practice  undermines  the  CRS 
observation:  “many  analysts  consider  the  increasing  willingness  of  agencies  to  voluntarily  take 
corrective  action  as  one  of  the  most  significant  trends  in  bid  protests.  Such  voluntary  action  by  an 
agency  could  indicate  that  the  agency  believes  that  a  given  protest  has  merit"  [emphasis  added] 
(Schwartz  &  Manuel,  2009,  p.  5).  This  increase  may  suggest  that  agencies  are  increasingly  making 
wrong  procurement  decisions  which  the  protests  are  forced  to  correct.  However,  clearly,  if  increases 
in  “voluntary  actions”  include  Fed-mail  and  Buy-off  settlements,  this  hardly  guarantees  that  those 
protests  have  merit.  On  the  other  hand,  as  described  by  Spriggs  and  Kidalov,  agencies  may  be 
engaging  in  risky  protest  litigation  and  incurring  unnecessary  legal  expenses  for  the  purpose  of 
protecting  agency  reputations  and  anti-competitive  practices.  As  a  result,  agencies  are  litigating 
protests  until  the  last  possible  moment  before  the  GAO  issues  a  formal  decision  even  where  “a 
reasonable  agency  inquiry  into  the  protest  would  show  facts  disclosing  the  absence  of  a  defensible 
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designed  to  reduce  disruptions  in  the  procurement  process  as  highlighted  in  the 
companion  report  to  this  study  (Benishek,  Sheinman,  Kidalov,  &  Angelis,  2011), 
suggest  that  the  buying  agencies  may  be  relying  on  the  GAO  bid  protest  process  as 
a  substitute  for  more  careful  acquisition  planning  and  source  selection. 

2.  It  Pays  to  Give  Procurement  Officials  the  Benefit  of  the  Doubt 

As  mentioned  earlier,  the  GAO  (and  most  procuring  agencies)  grants  an 
automatic  stay  upon  the  timely  filing  of  a  protest.  In  a  very  real  sense,  this  automatic 
shutdown  of  a  government  procurement  action  is  equivalent  to  assuming  that 
procurement  officials  are  guilty  until  proven  innocent.43  The  data  suggests  that  a 
powerful  argument  can  be  made  for  giving  procurement  officials  the  benefit  of  the 
doubt,  or  for  assuming  that  they  are  innocent  until  proven  guilty. 

Figure  1  displays  data  on  protest  outcomes  for  the  USAF  collected  from  the 
GAO’s  Bid  Protest  Docket,  as  reported  on  October  3,  2008.  Perhaps  the  most 
striking  feature  is  that  well  over  two  thirds  of  protests  in  this  sample  were  found  to  be 
without  merit  or  were  not  sustained.  Given  the  goal  of  promptly  and  efficiently 
completing  an  acquisition,  this  data  strongly  argues  against  forcing  agencies  to  incur 
potentially  costly  delays  as  a  result  of  the  suspension  of  a  procurement  (due  to  an 
automatic  stay)  while  a  protest  is  resolved. 


legal  position”  and  where  agencies  are  forced  to  incur  document  production  expenses  and  pay  the 
protester’s  legal  fees  (Spriggs  &  Kidalov,  2003). 

43  An  important  exception  is  the  Court  of  Federal  Claims  (COFC):  “In  contrast  to  the  automatic  stay  at 
the  GAO,  if  a  protester  files  its  protest  at  the  COFC,  it  must  meet  the  standards  for  a  preliminary 
injunction  to  obtain  a  stay.  Thus,  the  protester  must  demonstrate  that  it  is  likely  to  succeed  in  its 
protest,  it  will  suffer  irreparable  harm  unless  an  injunction  is  granted,  the  agency  will  not  suffer  a 
greater  harm  if  the  injunction  is  granted,  and  the  public  interest  favors  an  injunction”  (Schaengold  et 
al.,  2009,  p.  20).  Although  forcing  disappointed  bidders  to  seek  a  preliminary  injunction  to  stop  a 
procurement  is  more  costly  (and  risky)  to  the  protester,  it  is  recommended,  based  on  the  extremely 
low  rate  of  successful  protests  (and  the  substantially  higher  rate  of  protests  that  are  dismissed, 
denied,  etc.),  that  instead  of  an  automatic  stay,  agencies  and  the  GAO  adopt  the  COFC  standard  that 
procurement  officials  are  innocent  until  proven  guilty. 
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Undoubtedly  the  automatic-stay  procedure  benefits  some  contractors  by 
preserving  the  opportunity  for  resolution  of  a  meritorious  protest.  But  it  is  far  from 
clear ...  [whether]  these  benefits  outweigh  the  delays,  disruption,  and  increased 
costs  [imposed]  upon  legitimate  contract  awards,  particularly  in  light  of  the  fact  GAO 
denies  two  thirds  of  filed  protests.  (Metzger  &  Lyons,  2007,  p.  1240) 


Relative  Frequency  Histogram  of  GAO 
Bid  Protest  Docket  for  USAF  (10/3/08) 


Figure  1.  Rate  of  Unsuccessful  Protests — Almost  70%  Dismissed  or  Denied 

The  data  suggests  that  it  may  be  appropriate  (a)  to  overturn  the  implicit 
assumption  that  procurement  officials  (PO)  are  guilty  until  proven  innocent  that 
currently  underpins  the  GAO’s  automatic  stay  of  disputed  procurements,  and  (b)  to 
replace  it  with  the  assumption  (similar  to  that  of  the  Court  of  Federal  Claims)  that 
POs  are  innocent  until  proven  guilty.  Under  today’s  binding  GAO  rules,  and 
restrictive  regulations  and  statutes,  this  argues  for  departments  and  agencies  to 
consider  restricting  stays  (or  injunctions)  through  more  frequent  use  of  (stay 
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override)  arguments44  such  as  “urgent  and  compelling  circumstances”  or  “in  the 
interest  of  the  United  States”45  (see  FAR,  subpart  33.103;  FAR,  subpart  33.104). 

Although  rarely  addressed  in  the  literature,  this  argument  is  nicely  captured  in 
the  following  quotations  from  Metzger  and  Lyons  (2007,  p.  1238): 46 

With  CICA’s  automatic-stay  provision,  Congress  attempted  to  provide 
effective  review  of  bid  challenges,  and  in  the  process  to  encourage 
competition  in  contracting.  ...  By  staying  the  contract  pending  the  protest, 
CICA  implicitly  assumes  that  the  agency  is  at  fault.  ... 

A  losing  bidder  can  enjoin  a  competitor’s  contract  award  for  up  to  one 
hundred  days  simply  by  mailing  a  statement  to  the  Comptroller  General 
[GAO]  outlining  the  basis  of  its  protest.  ...  This  assumption  of  agency  fault 
and  ...  [the]  delay  prompted  by  a  GAO  stay  is  particularly  puzzling  given  the 
GAO  typically  sustains  only  one  third  of  protests  decided  on  the  merits.  This 
means  that  in  approximately  two  thirds  of  the  decided  protests  the  GAO  stay 
delays  legitimate  procurement  awards,  forcing  government  agencies  to 
extend  less  effective  legacy  contracts,  pursue  expensive  temporary  stopgap 
measures,  or  delay  the  functions  that  prompted  the  procurement,  (pp.  1238, 
1239) 


44  When  asked  how  often  their  agency  engages  in  defensive/preventive  strategies  in  order  to 
minimize  negative  impacts  of  bid  protests  on  the  acquisition  process,  over  70%  of  survey 
respondents  indicated  that  they  never  or  only  in  exceptional  cases  seek  overrides  of  mandatory  stays 
(Benishek  et  al.,  201 1 ,  pp.  67-68). 

45  Alternatively,  “By  adopting  a  more  rigorous  threshold  review  of  ...  filed  protests],  the  GAO  [could] 
limit  ...  procurement  actions  ...  subject  to  a  stay.  A  rapid  dismissal  of  ‘nuisance’  protests  [would 
avoid]  the  disruptive  and  costly  effects  of  an  unwarranted  stay.  ...  [Although  in  the  end,]  the 
Comptroller  General  [where  it  finds  an  agency’s  decision  unreasonable]  often  proceeds  directly  to 
invalidate  an  award,  or  require  re-competition.  ...  The  more  nuanced  COFC  approach  recognizes  that 
the  public  interest  is  not  always  served  by  delaying  a  procurement  to  correct  minor  defects  in  a 
solicitation  or  errors  in  the  conduct  of  a  competition”  (Metzger  &  Lyons,  2007,  p.  1 268).  Indeed,  the 
GAO  Bid  Protest  Regulations  presently  lack  even  the  standard  for  determining  whether  a  protest  is 
legally  frivolous  or  meritless. 

46A  more  balanced  system  exists  in  the  U.S.  Court  of  Federal  Claims  (COFC),  as  described  in  the 
companion  report,  Better  Acquisition  Management.  At  the  COFC,  a  stay  of  procurement  is 
discretionary  and  is  not  granted  if  the  government  can  establish  that  “(1 )  the  protester  is  not  likely  to 
succeed  on  the  merits  of  the  allegations  that  the  agency  action  was  illegal,  arbitrary,  capricious,  or 
abuse  of  discretion;  (2)  the  protester  will  not  suffer  irreparable  injury  without  injunctive  relief;  (3)  the 
balance  of  hardships  favors  the  government;  and  (4)  an  injunction  would  be  contrary  to  the  public 
interest,  including  considerations  of  national  defense  or  national  security”  (Benishek  et  al.,  201 1 ,  p. 
32).  Further,  under  COFC  Rules  65  and  65.1 ,  a  protester  must  post  a  bond  or  other  security 
sufficient  to  compensate  the  government  if  the  stay  was  ultimately  wrongful. 
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3. 


Defensive  Medicine:  The  Risk  of  Over-Deterrence 


Besides  the  strategic  use  of  protests  by  firms  to  extract  a  competitive 
advantage,  another  significant  cost  associated  with  the  current  protest  system  is  that 
acquisition  officials  risk  getting  caught  practicing  “defensive  medicine.”  Such  over¬ 
deterrence  is  manifested  in  a  proliferation  of  costly  (overlapping)  reviews  instituted  to 
ensure  that  the  design  and  conduct  of  procurement  actions  are  protest  proof.  This 
can  also  be  observed  in  the  behavior  of  procurement  officials  as  they  feel 
increasingly  paralyzed47  and  avoid  any  actions  that  might  risk  triggering  a  protest.48 

A  danger  is  that  acquisition  officials  will  become  more  concerned  with  input- 
focused  compliance  rather  than  output-focused  execution,  increasing  transaction 
costs  at  the  expense  of  procurement  outcomes.  Arrowsmith  et.  al.  (2000)  warns, 

when  the  procurement  system  is  primarily  rule-based,  the  focus  in  assessing 
the  performance  of  procurement  officers  tends  to  be  on  compliance  with  the 
rules  rather  than  with  the  quality  of  results  achieved.  ...  This  can  lead  to  a 
situation  where  even  within  the  area  of  discretion,  procurement  officers  may 
engage  in  unduly  cautious  behavior  in  order  to  avoid  possible  accusations  of 
non-compliance.  ... 

[Indirect  adverse  consequences  will  arise  as  a  result  of  a  fear  of  [protests] 
by  disappointed  contractors.  This  might  have  the  effect  of  inducing  undue 
deference  to  the  interests  of  contractors  to  the  detriment  of  the  public  interest. 
For  example,  public  authorities  may  become  ready  to  order  a  re-solicitation  at 


47  “Each  agency  should  preclude  management  layering  and  placing  nonessential  reporting 
procedures  and  paperwork  requirements  on  program  managers  and  contractors”  (Office  of 
Management  and  Budget  [OMB],  1976,  section  8). 

48  This  is  related  to  a  risk  recently  expressed  about  the  latest  Weapon  System  Acquisition  Reform  Act 
(WSARA)  of  2009: 

complaints  ...  often  heard  ...  [are]  that  the  WASRA  duplicates  existing  regulations,  adds  fresh 
layers  of  bureaucracy  and  piles  of  new  reporting  mandates  that  could  have  paralyzing  effects 
on  a  system  that  already  is  sluggish  and  unresponsive.  ...  A  combination  of  confusion  and 
fear  of  making  mistakes  is  fueling  paranoia  and  creating  an  environment  where  programs  will 
see  costs  rise  and  schedules  delayed.  ...  The  frustration  Congress  feels  translates  into  more 
reports,  and  more  oversight.  (Erwin,  2010) 

An  anonymous  official  recently  commented,  “Every  agency  has  over  200  years  of  governance  that 
tells  us  what  we  cannot  do”  (personal  communication). 
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the  slightest  complaint,  despite  the  delay  ...  rather  than  risk  legal  action  with 

its  potential  for  even  greater  disruption,  (pp.  76,  759) 

There  is  a  delicate  balance  between  the  desire  to  eliminate  any  hint  of  protest 
liability  in  order  to  ensure  absolute  compliance  to  avoid  protests,  and  the  risk  of 
growing  transaction  and  opportunity  costs  faced  by  the  department  to  move  an 
acquisition  forward.49  In  such  circumstances,  agencies  should  have  no  reluctance  to 
defend  their  decisions  or  to  use  ADR  and  other  expedited  resolution  tools  wherever 
appropriate. 

Another  serious  and  related  concern  is  the  risk  that  a  procurement  official 
(PO)  might  deny  one  company  an  award  in  order  to  avoid  incurring  protest  costs 
from  its  more  litigious  opponents.  Structuring  this  as  a  two-player  prisoner’s  dilemma 
game  suggests  that  any  hint  of  favoritism  on  the  part  of  a  PO  would  likely  increase 
every  bidder’s  propensity  to  protest. 

The  logic  is  straightforward.  If  one  firm  is  more  inclined  to  protest  and  the 
other  is  not,  then  the  protesting  firm  increases  its  probability  of  winning  at  the 
expense  of  the  other  firm.  Since  the  game  is  symmetric,  both  firms  will  eventually  be 
more  inclined  to  protest  or  risk  being  shut  out  of  future  competitions. 

The  result  is  similar  to  an  arms  race  competition  in  the  sense  that,  everyone 
is  more  likely  to  protest  and  yet  no  one  is  any  more  secure  in  obtaining  a  contract. 
This  might  help  explain  a  recent  observation  by  current  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  &  Logistics  Ashton  Carter:  “I  am  and  the  entire 
department  is  concerned  about  protests  becoming  common  or  routine”  (as  cited  in 


49  The  risk  is  that  the  protest  system  “tail”  begins  to  wag  the  procurement  process  “dog  to  the 
detriment  of  procurement  outcomes.  This  risk  is  amplified  if  officials  are  held  accountable  for  current 
delays  from  protests,  but  benefit  only  indirectly  from  successful  future  outcomes — i.e.  final  fielding  of 
future  programs  or  capabilities.  The  latter  issue  relates  to  the  principal-agent  problem  discussed  later, 
where  alternative  ways  of  achieving  the  procurement  objectives  of  equity,  integrity,  and  efficiency  are 
also  explored. 
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Shalal-Esa,  2009).  A  companion  explanation  emerges  when  placed  in  the  shoes  of  a 
disappointed  bidder,  and  asked  to  look  forward  and  reason  back. 

4.  Placing  Ourselves  in  the  Shoes  of  a  Disappointed  Bidder 

In  order  to  understand  a  firm’s  protest  strategy,  the  economics  approach 
suggests  looking  forward  and  reasoning  back.  This  involves  three  steps  that  result  in 
two  key  recommendations. 

(1 )  The  first  step  is  to  place  ourselves  in  the  shoes  of  a  disappointed 
bidder  and  ask,  what  are  their  goals  and  objectives?  The  answer  is 
likely  profit  maximization  (i.e.,  a  return  to  shareholders),  including  the 
immediate  contract  and  any  follow-on  business.  Much  like  playing  the 
lottery,  a  disappointed  bidder  who  maximizes  profits  will  calculate  the 
expected  value  of  winning  the  protest.  The  expected  benefit  is  the 
probability  of  winning  the  protest  (i.e.,  the  likelihood  a  protest  is 
sustained)  multiplied  by  the  payoff  (i.e.,  the  reward).  A  simplified 
version  of  a  disappointed  bidder’s  decision  problem  is  illustrated  in 
Figure  2.  To  determine  whether  to  file  a  protest,  the  disappointed 
bidder  looks  forward  and  reasons  back.  The  expected  benefit  of  filing  a 
protest  is  roughly  the  probability  that  the  GAO  will  sustain  the  protest 
(Ps)  multiplied  by  the  payoff. 


Figure  2.  Disappointed  Bidder  Decision  Tree 
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(2)  The  second  step  is  to  place  ourselves  in  the  shoes  of  a  disappointed 
bidder  and  ask,  what  constrains  their  behavior?  Unlike  playing  the 
lottery  for  fun,  economists  assume  that  CEOs  are  rational  “players” 
constrained  by  shareholders/owners  and  are  likely  to  engage  in  a 
benefit-cost-type  analysis  to  determine  whether  to  play  (i.e.,  to  file  a 
protest).  In  making  its  protest  decision,  a  company  will  weigh  the 
discounted  present  value  (or  option  value)  of  the  expected  benefits  of 
playing  a  protest  lottery  against  the  costs.  The  higher  the  cost  of 
playing,  the  less  likely  a  firm  is  to  file  a  protest.  In  this  case,  “buying  a 
ticket”  to  play  (i.e.,  filing  a  protest)  can  include  filing  fees,  hiring  lawyers 
and  subject-matter  experts,  paying  penalties,  and  the  significant 
opportunity  costs  of  diverting  the  company’s  focus  and  resources.  The 
expected  costs  of  filing  a  protest  are  roughly  the  probability  that  the 
GAO  will  deny  the  protest  (1-Ps)  multiplied  by  any  penalties  (including 
opportunity  costs).50 

(3)  The  third  and  final  step  is  to  place  ourselves  in  the  shoes  of  officials 
representing  the  procurement  activity.  This  involves  asking  how  we 
can  align  the  interests  of  a  contractor  with  the  defense  activity’s 
interests  to  reduce  protests.  The  first  opportunity  is  to  lower  the 
expected  benefits  of  a  protest  (i.e.,  lower  the  probability  of  success 
and/or  the  payoff).  The  second  opportunity  is  to  increase  the  expected 
costs. 

The  economics  approach  automatically  generates  two  recommendations  to 
reduce  protests:  lower  the  expected  benefits  of  a  protest  or  increase  the  expected 
costs. 


Options  to  increase  the  expected  costs  of  a  protest,  include:  (i)  filing  fees 
could  be  raised  to  reflect  the  government’s  (agency  and  GAO)  complete  costs  to 
administer  a  protest;  (ii)  firms  could  be  required  to  post  a  bond  to  compensate  for 
delays  in  the  event  that  the  protest  is  denied;  or  (iii)  penalties  might  be  assessed  in 


50  The  detailed  model  can  be  found  in  Section  III  of  this  study. 
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the  event  of  protests  which  are  either  clearly  strategic  and  not  in  the  public  interest, 
or  largely  frivolous  protests. 

To  lower  expected  benefits,  the  first  step  is  to  identify  feasible  ways  to  reduce 
the  probability  of  successful  protests.  For  example,  fraud  and  errors  could  be 
reduced  by  (i)  increasing  training  and  experience,  (ii)  ensuring  the  transparency  of 
the  assessment  criteria,  (iii)  ensuring  the  transparency  and  accountability  of  the 
evaluation  and  selection  process,  and  (iii)  making  companies  aware  of  the  low 
probability  of  awards  being  overturned 

The  second  step  is  to  identify  feasible  ways  to  reduce  the  payoff  from  a 
successful  protest.  Examples  of  how  this  could  be  done  include  (i)  reducing  the 
award  (unbundling  the  contract,  etc.),  and  (ii)  keeping  records  of  protests  by  firms 
and  using  this  information  in  future  competitions  (i.e.,  a  reputation  for  delaying 
projects  through  frivolous  protests),  This  latter  option,  however,  could  be 
unconstitutional,  against  the  First  and  Fifth  Amendments,  unless  targeted  protests 
are  truly  frivolous  as  defined  by  Federal  judicial  precedents. 

Another  interesting  insight  generated  by  the  economics  approach  is  that 
there  may  be  no  such  thing  as  a  frivolous  protest;  only  strategic  protests.  A  rational, 
calculating  disappointed  bidder  will  only  file  a  protest  if  the  expected  benefits 
outweigh  the  expected  costs.  Therefore,  in  reality,  so-called  “frivolous”  lawsuits  likely 
reflect  the  strategic  use  of  protests  by  firms.  A  major  challenge  for  a  procuring 
activity  is  to  reap  the  benefits  of  protests  while  limiting  the  unintended 
disruptions/consequences  caused  by  the  protest  process.51 


51  Because  the  Defense  Department  is  grappling  with  this  challenge,  it  must  avoid  retaliation  in  strict 
compliance  with  settled  constitutional  principles  of  freedom  to  petition  the  government  for  redress  of 
grievances  and  due  process  under  the  First  and  Fifth  Amendments.  It  is  critical  to  make  distinctions 
between  legally  frivolous  protests  that  can  be  sanctioned  under  law  and  protests  that  are 
unmeritorious,  marginally  meritorious,  or  insufficiently  meritorious  for  injunctive  relief.  The  GAO 
considers  a  protest  to  be  “clearly  meritorious”  when  “a  reasonable  agency  inquiry  into  the  protester’s 
allegations  would  show  facts  disclosing  the  absence  of  a  defensible  legal  position”  GAO,  The  Real 
Estate  Center-Costs,  B-274081.7  (1998).  However,  as  noted  in  the  companion  Benishek  et  al. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  -  28  - 

NAVAL  POSTGRADUATE  SCHOOL 


B.  The  Optimum  Rate  of  Protests 


The  effort  to  balance  the  benefits  of  protests  against  the  potentially  significant 
(opportunity  and  transaction)  costs  suggests  that  there  is  an  optimum  rate  of 
protests.  Although  it  is  unrealistic  to  think  that  we  can  precisely  pin  down  this 
optimum,  it  is  useful  to  discuss  its  characteristics. 

For  example,  suppose  the  Department  of  Defense  finds  itself  above  the 
optimum.  In  this  case,  reducing  the  rate  of  protests  is  warranted  because  benefits  to 
all  parties  (buyers,  sellers,  and  taxpayers)  outweigh  the  costs.  However,  if  the 
Defense  Department  happens  to  be  at  or  below  the  optimum  rate,  then  any  direct 
(short-run)  benefits  from  reducing  protests  could  be  more  than  offset  by  indirect 


(2011,  pp.  111-112)  report,  the  GAO  presently  does  not  make  a  determination  of  whether  a  protest  is 
legally  frivolous,  but  the  COFC  does: 

Under  Rule  1 1  of  the  Rules  of  the  Court  of  Federal  Claims,  a  protester  can  be  sanctioned 
only  if  the  protest:  (1)  is  being  presented  for  any  improper  purpose,  such  as  to  harass,  cause 
unnecessary  delay,  or  needlessly  increase  the  cost  of  litigation;  (2)  the  claims,  defenses,  and 
other  legal  contentions  are  warranted  by  existing  law  or  by  a  nonfrivolous  argument  for 
extending,  modifying,  or  reversing  existing  law  or  for  establishing  new  law;  (3)  the  factual 
contentions  have  evidentiary  support  or,  if  specifically  so  identified,  will  likely  have  evidentiary 
support  after  a  reasonable  opportunity  for  further  investigation  or  discovery;  and  (4)  the 
denials  of  factual  contentions  are  warranted  on  the  evidence  or,  if  specifically  so  identified, 
are  reasonably  based  on  belief  or  a  lack  of  information. 

Under  Rule  11, 

a  sanction  imposed  under  this  rule  must  be  limited  to  what  suffices  to  deter  repetition  of  the 
conduct  or  comparable  conduct  by  others  similarly  situated.  The  sanction  may  include 
nonmonetary  directives;  an  order  to  pay  a  penalty  into  court;  or,  if  imposed  on  motion  and 
warranted  for  effective  deterrence,  an  order  directing  payment  to  the  movant  of  part  or  all  of 
the  reasonable  attorney’s  fees  and  other  expenses  directly  resulting  from  the  violation.  (Rules 
of  the  Court  of  Federal  Claims,  Rule  1 1  as  amended  through  Jan.  1 1 , 201 0).  CITATION) 

The  COFC  does  not  sanction  protests  that  do  not  meet  these  requirements.  As  noted  in  the 
Better  Acquisition  Management  report  “Necessarily,  a  protestor  would  have  some  right  to  due 
process  in  order  to  “avoid  punishing  a  company  for  filing  a  good-faith  but  unmeritorious  protest.” 
(GAO,  B-401197,  2009,  13).  This  process  may  actually  make  the  overall  protest  process  longer, 
which  could  have  potential  negative  impacts  on  the  acquisition  system.  However,  bringing  the  GAO 
Bid  Protest  Regulations  to  parity  with  COFC  concerning  sanctions  for  truly  frivolous  protests  could 
provide  a  deterrent  and,  more  importantly,  address  the  perception  that  GAO  may  be  welcoming 
frivolous  protests.”  However,  the  COFC  Rules  65  and  65.1  require  protesters  in  unmeritorious 
protests  to  forfeit  bonds  for  wrongfully  restraining  the  government.  Further,  under  title  28,  section 
1491(b)(3),  the  COFC  can  deny  a  permanent  injunction  remedy  where  the  agency’s  conduct  was 
arbitrary  or  illegal,  but  the  illegality  or  arbitrariness  is  outweighed  by  national  security,  national 
defense,  or  other  public  interest  considerations. 
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(long-run)  costs — e.g.,  undermining  the  deterrent  against  errors,  fraud,  and 
favoritism.  In  this  case,  the  net  effect  of  reducing  protests  could  turn  out  to  be 
detrimental  to  the  long-term  interests  of  the  Defense  Department.  It  is  enlightening  to 
explore  what  is  an  understandable  aim  of  the  Defense  Department:  to  reduce  the 
rate  of  successful  protests. 

1.  Exploring  the  Objective  of  Reducing  the  Rate  of  Successful 
Protests 

In  discussing  the  DoD’s  objective  to  reduce  the  rate  of  successful  protests,  it 
is  important  to  recognize  that  the  impacts  of  protests  on  a  procuring  activity  vary 
greatly.  Policies  designed  to  reduce  the  rate  of  successful  protests  can  either  focus 
on  acquisitions  that  have  a  relatively  trivial  impact  on  the  activity  or,  instead,  on 
those  that  are  critical  to  (or  on  the  critical  path  of)  key  activity  investments  and 
operations.  Clearly,  scarce  leadership  time  and  resources  are  better  spent  on  the 
latter  (critical,  major  acquisitions)  rather  than  on  the  former  (non-critical,  minor 
acquisitions). 

A  simple  illustration  of  this  important  distinction,  using  data  from  the  USAF, 
appears  in  Figure  3.  Over  a  recent  eight-year  period  from  FY2001-2008,  even 
though  the  rate  of  successful  protests  or  the  “sustain  rate”  was  relatively  constant, 
the  magnitude  of  those  protests,  or  the  “impact  rate”  (measured  here  in  terms  of  the 
share  of  competitive  procurement  dollars  involved),  increased  dramatically. 

If  senior  leaders  were  to  strictly  focus  on  reducing  the  sustain  rate  (or  the  rate 
of  successful  protests),  then  since  the  USAF  only  faced  3  successful  protests  in 
2001  out  of  the  98  protests  of  procurements  closed  that  year,  it  faced  a  sustain  rate 
of  nearly  3%.  From  this  perspective,  the  protest  problem  appears  to  be  stable  or 
even  declining;  there  were  only  2  successful  protests  out  of  1 12  closed  in  2008,  a 
sustain  rate  just  under  2%  (see  Figure  3).  Unfortunately,  simply  focusing  on  the  rate 
of  successful  protests  (the  sustain  rate)  is  misleading.  It  hides  the  true  impact  of 
those  protests  on  the  USAF. 
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It  turns  out  that  in  2001 ,  successful  protests  against  the  USAF  involved 
roughly  $260  million  worth  of  procurements,  or  an  impact  rate  of  around  1  %  of  total 
competitive  procurement  dollars  that  year.  Compare  this  to  a  contract  amount  of 
over  $36  billion  tied  up  in  the  two  successful  protests  in  2008,  or  an  impact  rate  of 
nearly  97%  of  USAF  competitive  procurement  dollars  (see  Figure  3).  It  is  clear  that 
simply  focusing  on  reducing  the  rate  of  successful  protests  (the  sustain  rate)  risks 
missing  the  larger  impact  those  protests  have  on  USAF  investments  and  operations. 


Impact  Rate  = 

Sustain  Value/Contract 
Value 


Sustain  Rate  = 
Procurements 
Sustained/Procurements 
Closed 


Figure  3.  Rate  of  Successful  Protests — Sustain  Rate  versus  Impact  Rate 

(Data  is  from  USAF  Annual  Bid  Protest  Update  [January  1,  2009]:  Synopsis  of  Bid 

Protests  [GAO]  for  FY200 1-2008) 

In  shaping  protest  policy,  senior  leadership  should  focus  its  attention  on 
critical  procurements — i.e.,  those  acquisitions  likely  to  have  the  greatest  impacts  on 
investments  and  operations  if  they  are  protested — possibly,  but  not  necessarily,  high 
cost  Major  Defense  Acquisition  Programs.  It  could  be  useful  for  each  Service  to 
develop  a  measure  similar  to  the  “impact  rate”  illustrated  in  Figure  3.  To  help  shape 
its  protest  policies,  this  study  recommends  that  the  Defense  Department  focus  on 
identifying  key  factors  and  characteristics  associated  or  correlated  with  successful 
protests  of  critical,  high-impact  procurements. 
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2. 


How  Reducing  Protests  Can  Actually  Increase  Competition 


The  assumption  captured  in  the  opening  quotation  of  the  Introduction — an 
assumption  that  appears  throughout  the  legal  and  regulatory  literature — is  that 
protests  act  as  a  deterrent  to  bad  behavior  on  the  part  of  procurement  officials.  The 
general  conclusion  in  the  literature  is  that  increasing  the  ability  to  protest  is  likely  to 
increase  competition.  However,  from  an  economics  perspective,  this  implicitly 
assumes  we  are  below  the  optimum  level  of  protests — or  that  the  social  benefits  of 
increasing  protests  outweigh  the  social  costs. 

The  Defense  Department  is  concerned  with  the  opposite — reducing  the  rate 
of  protests.  But  this  implicitly  assumes  that  we  are  above  the  optimum  level  of 
protests,  or  that  the  benefits  of  reducing  protests  outweigh  the  costs.  Fortunately, 
there  are  two  ways  to  reduce  protests  that  could  actually  increase  competition, 
regardless  of  where  the  Defense  Department  happens  to  be  relative  to  the  optimum, 
and  conceivably  improve  procurement  outcomes. 

Rather  than  simply  adopting  a  popular  objective  such  as  minimizing  the 
number  of  protests  (see  Schwartz  &  Manuel,  2009,  p.  1),  a  more  nuanced 
economics  view  of  the  protest  problem  is  based  on  three  fundamental  assumptions: 
(i)  incentives  matter,  (ii)  prices  matter,  and  (iii)  demand  curves  slope  down.  Given 
these  assumptions,  and  placing  ourselves  in  the  shoes  of  a  representative 
prospective  bidder,  we  obtain  a  powerful,  new,  and  somewhat  counterintuitive 
insight:  Regardless  of  where  the  Defense  Department  happens  to  be  relative  to  the 
optimum,  there  exist  policies  to  reduce  protests  that  can  actually  increase 
competition. 

The  two  main  explanations  for  this  include  one  that  is  obvious  and  one  that  is 
less  obvious.  The  first  involves  the  possibility  of  reducing  protests  by  decreasing  the 
expected  benefits  to  a  disappointed  bidder  of  challenging  a  procurement  action.  The 
second  involves  reducing  protests  by  increasing  the  expected  costs  of  a  challenge. 
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(4)  Decreasing  the  expected  benefits  of  a  challenge:  The  first,  somewhat 
obvious,  explanation  of  how  reducing  protests  can  increase 
competition  focuses  on  reducing  protests  by  reducing  agency  errors, 
fraud,  and  favoritism — for  example,  by  substituting  protests  with 
alternatives  such  as  random  audits  and  investigations  or  by  investing 
in  training,  linking  pay  and  promotion  to  successful  procurement 
outcomes,  and  hiring  more  experienced  acquisition  personnel.52  This 
approach  reduces  protests  but  simultaneously  tends  to  build 
confidence  in  the  integrity  of  the  solicitation  and  selection  process, 
boosting  the  expected  benefits  of  participation,  and,  therefore,  is  likely 
to  increase  competition. 

(5)  Increasing  the  expected  costs  of  a  challenge:  There  is  another  less 
obvious  explanation  where  reducing  protests  can  increase  competition. 
Even  where  protests  are  reduced  by  raising  the  expected  costs  of  a 
challenge — for  example,  by  increasing  filing  fees,  restricting  the  filing 
window,  abolishing  mandatory  stay,  narrowing  standing  (i.e. ,  eligibility), 
raising  the  threshold  for  a  protest  to  have  merit,53  or  establishing 
penalties  such  as  fines  or  exclusions  from  bidding  or  filing  future 
protests  for  truly  frivolous/nuisance  protests  and/or  “loser  pays”  for 
protests  that  have  some  merit  but  are  not  sustained — this  could  still 
increase  competition.  In  this  case,  the  explanation  is  less 
straightforward,  but  only  because  it  is  largely  overlooked  in  the  current 
legal  and  regulatory  literature. 


52  The  Joint  Analysis  Team  (JAT),  organized  by  the  Under  Secretary  of  Defense,  has  collected  data 
that  indicates  acquisition  workloads  recently  increased  while  the  level  of  experienced  personnel 
decreased.  The  data  reveals  that  the  DoD  has  seen  a  significant  drop  in  its  trained,  mid-level 
acquisition  personnel.  Over  the  period  FY1998-FY2008,  those  with  10  to  19  years  of  experience 
declined  from  47%  of  the  workforce  to  17%  (JAT,  2009). 

53  Raising  the  threshold  for  merit  involves  Type  I  and  II  errors  from  statistical  hypothesis  testing. 
Suppose  the  null  hypothesis  is  that  the  agency  (PO)  is  innocent.  Increasing  the  burden  of  proof  that 
an  agency  is  guilty  decreases  the  Type  I  error  (reject  null  when  null  is  true),  but  increases  Type  II 
errors  (accept  null/innocent  when  null  is  false,  i.e.,  guilty).  However,  the  cost  to  an  agency  or 
department  is  substantial  given  the  automatic  stay  assumption  of  guilty  until  proven  innocent  and 
stands  in  sharp  contrast  to  the  demanding  standards  required  for  a  preliminary  injunction  (innocent 
until  proven  guilty)  at  the  Court  of  Federal  Claims. 
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The  primary  focus  of  the  extant  literature  is  on  so-called  prospective  “losing” 
bidders.  Clearly,  bidders  that  fear  losing  a  competition  are  more  inclined  to 
participate  the  greater  the  probability  they  might  get  a  second  bite  at  the  apple  (i.e., 
the  greater  the  opportunity  to  protest  a  negative  outcome).  So  while  the  procuring 
activity  could  lower  its  expected  costs  of  a  protest  by  promoting  policies  that 
increase  a  bidder’s  expected  costs  of  engaging  in  a  protest,  since  this  increases 
risks  faced  by  “losing”  bidders,  if  they  refuse  to  participate  in  the  bidding  it  could 
harm  competition.  The  conclusion  drawn  in  the  legal  and  regulatory  literature  is  that 
reducing  the  ability  to  protest  is  likely  to  reduce  competition. 

However,  the  literature  is  nearly  silent  when  it  comes  to  another  important 
type  of  bidder:  the  so-called  prospective  “winning”  bidders.  The  greater  the  risk  (and 
expected  costs)  of  protests,  the  less  inclined  prospective  winning  bidders  will  be  to 
participate  in  a  competition.54  In  focusing  on  losing  bidder  types,  the  literature 
ignores  the  role  played  by  winning  bidder  types.  From  a  winning  bidder’s 
perspective,  a  protest  delays  the  project  and  generates  significant  (opportunity) 
costs  to  defend  an  award — similar  in  nature  to  delays  and  costs  incurred  by  the 
procuring  activity.55  In  this  case,  policies  that  restrict  protests  could  actually  increase 
competition,  since  limiting  protests  generates  a  greater  expected  prize  that  (on  the 
margin)  should  attract  more  prospective  winning  bidder  types. 

In  summary,  by  selecting  policies  that  reduce  the  protest  risks  of  procurement 
activities,  the  DoD  might  increase  the  risks  faced  by  losing  bidder  types,  hurting 
competition,  but  it  could  also  reduce  the  risks  faced  by  winning  bidder  types, 


54  In  other  words,  they  will  charge  (bid)  a  higher  price  to  compensate  for  this  extra  risk  to  participate  in 
a  competition.  Publicly  traded  government  contractor  firms  are  obligated  to  account  for  bid  protest 
risks  to  their  investors  as  part  of  so-called  “forward-looking  statements”  and  disclaimers  under  the 
Private  Securities  Litigation  Reform  Act  (PSLRA)  of  1995  (1995).  Conversely,  a  disappointed  bidder 
may  introduce  the  element  of  risk  for  winning  bidders  by  publicly  announcing  that  it  intends  to  protest 
and  tagging  the  threat  of  protest  as  a  PSLRA  statement.  The  threat  of  protests  (even  if  unsuccessful) 
can  act  as  barriers  to  entry.  The  prospect  of  a  protest  by  a  losing  bidder  imposes  costs  that  reduce 
the  expected  value  of  entering  the  competition  (see  the  representative  bidder  model  in  Section  III). 

55  As  pointed  out  in  the  literature,  “In  post-award  protests,  awardees  will  often  intervene  on  the  side  of 
the  agency  to  protect  their  award”  (MMR,  1991,  p.  14). 
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boosting  competition.  The  net  effect  is  an  empirical  question  with  enormous  policy 
implications.  A  recommendation  of  this  study  is  for  procurement  officials  to  sponser 
an  empirical  analysis  to  investigate  this  issue  (including  an  estimate  of  any  bid 
protest  premiums  incurred  by  contractors  on  capital  markets  and  passed  on  to  the 
government). 

It  could  turn  out  that  the  net  effect  on  competition  depends  on  where  an 
activity  finds  itself  in  terms  of  the  optimum  rate  of  protests.  For  instance,  if  in  some 
range  limiting  protests  by  raising  the  expected  costs  of  a  challenge  actually 
increases  competition,  then  this  could  mitigate  any  perceived  negative  effects  that 
reducing  protests  has  on  the  integrity,  transparency,  and  accountability  of  the 
procurement  process. 

Ideally,  policy  initiatives  that  limit  protests  could  be  combined  with  a  portfolio 
of  other  carefully  selected,  cost-effective  governance  mechanisms  (internal  audits, 
external  audits,  independent  investigations,  training  investments,  etc.),  reducing  the 
need  for  the  monitoring  and  oversight  role  of  protests.  In  terms  of  limiting  protests, 
an  extreme  example  is  found  in  the  private  sector.  Protest  systems  are  virtually 
unheard  of  in  the  broader  economy. 

C.  Public  versus  Private  Procurement  Governance 
Mechanisms:  A  Counter-Factual 

In  sharp  contrast  to  government  protest  systems,  the  private  sector  rarely 
allows  bid  protests.  Instead,  private  firms  govern  their  transactions  using  a  different 
set  of  oversight  mechanisms.  They  rely  more  on  reputation,  internal  audits,  exacting 
definitions  of  buyer  requirements  (see  generally,  Johnson,  2003),  external 
(shareholder)  audits  and  on  aligning  the  incentives  of  procurement  personnel  with 
those  of  the  company  through  performance-based  bonuses  and  promotions. 

Under  private  contract  law,  disappointed  offerors  generally  have  no  right  to 
attack  the  buyer’s  choice  of  suppliers.  Private  contract  law  usually  provides 
no  basis  for  recovery  where  the  offeror  contends  that  the  buyer:  (1 )  defined 
specifications  of  a  good  or  service  in  a  manner  that  precluded  consideration 
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of  the  offeror’s  goods,  (2)  refused  to  consider  the  bids  of  specific  offerors,  (3) 
failed  to  adhere  to  announced  selection  criteria,  or  (4)  applied  its  selection 
criteria  unevenly.  (Kovacic,  1995,  p.  466) 

The  profit  motive  helps  align  the  interests  of  contracting  personnel  to  solicit 
and  select  contractual  partners  that  maximize  value  to  the  firm.  The  powerful 
oversight  role  of  market  forces  automatically  rewards  companies  that  select  vendors 
who  offer  the  best  value-added  combination  of  performance,  costs,  and  schedules. 
Companies  with  poor  contractual  outcomes  are  punished  with  losses  by  market 
forces.  They  are  eventually  broken  up  or  go  bankrupt.  In  fact,  a  firm’s  contracting 
reputation  is  key  to  its  successful  interactions  with  other  companies. 

The  private  contract  system  relies  mainly  on  reputation  to  constrain  the 
discretion  of  private  purchasers.  ...  A  private  buyer  cannot  afford  to  treat 
suppliers  of  inputs  too  arbitrarily  or  unfairly,  lest  sellers  refuse  to  deal  with  the 
buyer  in  the  future.  (Kovacic,  1995,  p.  466) 

In  addition,  unfairly  treated  suppliers  may  have  remedies  under  the  Federal  or 
state  antitrust  laws  (see,  for  example,  Federal  Trade  Commission,  2010).  Courts 
have  carefully  limited  these  remedies  to  avoid  granting  advantage  to  individual 
competitors  at  the  expense  of  the  public  interest  in  competition.  For  example, 
remedies  for  attempted  monopolizing  of  trade  in  violation  of  Section  2  of  the 
Sherman  Act  (2010).  “should  not  attempt  to  redress  harm  to  competition  by 
‘providing  aid  to  any  particular  competitor,’  but  rather  should  aim  ‘to  restore 
conditions  in  which  the  competitive  process  is  revived  and  any  number  of 
competitors  may  flourish  (or  not)  based  upon  the  merits  of  their  offerings’” 
(Department  of  Justice,  2008).  As  a  result,  antitrust  remedies  for  disappointed 
bidders  in  private  sector  transactions  are  significantly  weaker  than  bid  protest 
remedies  for  transactions  with  the  public  sector. 

Transaction  Cost  Economics  (TCE)  suggests  that  cooperative  behavior  often 
arises  in  situations  of  repeated  private  interaction  in  which  each  side  has  a  sufficient 
enough  stake  in  the  future  to  outweigh  any  short-term  gains  from  acting 
opportunistically  (Melese  et  al .,  2007).  Either  a  relationship  must  be  repeated 
frequently,  or  the  value  of  continuing  the  relationship  must  be  high.  As  long  as 
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performance  can  be  easily  monitored  by  future  trading  partners,  reputation  can 
govern  behavior.  However,  the  noisier  (more  complex/uncertain)  the  product, 
service,  or  the  environment,  the  less  effectively  this  works.  As  complexity  increases 
and  frequency  falls,  relational  contracting  tends  to  evolve  from  a  bilateral  relationship 
to  a  trilateral  relationship. 56 

In  this  case,  a  third  party  (independent  auditor,  alternative  dispute  resolution 
(ADR)  agreement,  etc.)  often  provides  guarantees  to  facilitate  and  improve 
contractual  outcomes.  The  third  party  can  help  monitor  transactions  or  offer 
arbitration  when  disputes  arise. 

Bid  protest  systems  are  a  type  of  third-party  monitoring  mechanism.  However, 
it  is  sobering  to  consider  what  the  ramifications  would  be  if  the  current  federal 
protest  process  was  forced  upon  the  private  sector.  It  would  likely  shut  down  entire 
industries. 


56  An  important  insight  of  the  Transaction  Costs  Economics  (TCE)  literature  (see  Melese  et  al. ,  2007; 
Williamson,  1971,  1979,  1999)  is  that  ex-ante  competition  can  lead  to  ex-post  monopoly  power. 
According  to  basic  TCE  principles,  there  is  always  a  risk  that  firms  in  a  transaction  will  act 
opportunistically.  The  hold-up  problem  is  a  classic  example.  Related  to  the  foot-in-the-door  strategy,  a 
firm  might  strategically  bid  low  to  win  a  contract  with  the  aim  of  raising  prices  later.  After  winning  the 
contract,  if  a  firm  invests  in  specific  assets  (specialized  human  capital  or  equipment,  a  location 
advantage,  etc.)  that  make  that  firm  more  valuable  in  terms  of  the  transaction  (performance,  cost,  and 
schedule),  but  that  also  act  as  a  barrier  to  future  entry  by  competitors,  then  it  acquires  the  ability  to 
hold-up  the  government.  Since  one  of  the  stated  goals  of  a  protest  system  is  to  increase  competition, 
this  is  an  important  consideration.  The  risk  is  that  valuable  investments  in  assets  that  are  specific  to 
the  relationship  can  change  the  nature  of  that  relationship  from  one  of  ex-ante  competition  to  ex-post 
monopoly  supplier.  In  this  case,  the  firm  has  the  ability  to  threaten  to  hold-up  the  government  and 
raise  prices  (say  on  change  orders,  etc.)  high  enough  to  more  than  make  up  for  its  artificially  low 
initial  bid.  It  would  be  interesting  to  determine  empirically  if  the  hold-up  problem  merits  more  attention 
than,  say,  is  currently  being  paid  to  bid  protests  in  the  public  procurement  selection  process. 
Regardless  of  who  wins,  if  data  suggests  that  hold-up-type  problems  routinely  contribute  to  schedules 
and  performance  slipping  and  prices/costs  increasing,  then  winning  bid  proposals  are  not  credible. 
Since  losing  firms  no  longer  have  an  option  to  protest,  a  recommendation  might  be  to  consider 
building  a  clause  into  contracts  that  compensates  the  losing  bidder  or  interested  party  (i.e.,  the  first 
runner-up)  in  the  event  that  the  winning  bidder  cannot  perform  what  they  promised  in  order  to  win  the 
competition.  The  goal  is  to  hold  companies  accountable  for  following  through  with  proposals  offered 
to  win  a  contract.  The  outcome  could  be  more  truthful  bids/proposals  and  better  outcomes  for  our 
troops  and  taxpayers. 
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Evidence  that  imposing  the  public  protest  process  on  the  private  sector  would 
be  prohibitively  costly  is  strongly  reinforced  by  the  following  example.  This  example 
involves  a  relationship  between  three  private  defense  contractors  and  their 
suppliers/subcontractors. 

To  place  the  potentially  crushing  costs  of  a  private  bid  protest  process  in 
perspective,  consider  the  findings  of  a  1992  Commerce  Department  study  that 
examined  the  supply  chain  for  three  representative  U.S.  Naval  weapon  systems:  the 
HARM  missile,  the  Mark-48  ADCAP  torpedo,  and  the  Verdin  communication  system. 
It  turns  out  that  a  total  of  “1 1 ,638  companies  . . .  [served]  as  active  suppliers  to  the 
prime  contractors  for  the  three  weapon  systems,”  6,818  for  the  HARM  missile,  1,483 
for  the  Verdin,  and  3,336  for  the  Mark-48  torpedo  (Department  of  Commerce, 

1992). 57 


Forced  to  abide  by  the  government’s  bid  protest  process  in  dealing  with  their 
multiple  suppliers  and  subcontractors,  these  companies  would  have  incurred 
prohibitively  high  transaction  costs.  The  protest  process  would  have  imposed 
significant  delays,  cost-overruns,  and  performance  gaps  that  would  have  forced 
much  of  the  work  in-house  or  shut  down  operations  entirely.  The  likely  result:  lower 
performance,  higher  costs,  and  significant  delays  in  the  three  weapons  systems. 
This  sobering  example  should  place  the  considerable  burden  of  a  bid  protest 
process  in  perspective. 


57  These  are  not  unusual  cases  in  the  defense  sector.  There  are  strong  (“rent-seeking”)  incentives  to 
proliferate  suppliers  across  political  jurisdictions  in  order  to  increase  the  probability  of  favorable 
treatment  in  congressional  appropriations.  For  example,  the  Air  Force  B-1  program  had 
subcontractors  in  49  states,  and  the  B-2  had  subcontractors  in  all  50. 
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So  why  does  the  protest  process  dominate  public  procurement  law58  and  give 
disappointed  bidders/offerors  such  expansive  rights  to  challenge  an  acquisition?  The 
answer  lies  in  the  economist’s  principal-agent  problem.59 

D.  The  Principal-Agent  Problem 

The  delegation  of  procurement  authority  raises  a  principal-agent  problem.  In 
the  simplest  terms,  the  government  (the  principal),  who  represents  troops  and 
taxpayers,  asks  procurement  officials  (agents)  to  conduct  procurement  actions  on  its 
behalf.  The  principal-agent  problem  arises  if  there  are  conflicting  objectives  (i.e.,  if 
agents  do  not  have  the  same  objectives  as  the  principal)  and  if  key  aspects  of  the 
agent’s  behavior  are  not  easily  monitored  (also  known  as  the  agency  problem).60 

A  serious  issue  in  the  public  sector  is  that  the  reward  system  does  not  encourage  a 
high  level  of  effort  (training  investment  or  experience)  and  does  not  necessarily  align 
the  interests  of  the  agent  with  the  principal.  The  risk  is  that  the  principal’s  (taxpayers 
and  troops)  objectives  of  better,  faster,  cheaper  may  not  be  fully  adopted  by  agents 
(procurement  officials)  responsible  for  carrying  out  those  decisions:  .  “Government 


58  Two  arguments  occasionally  heard  that  help  explain  the  willingness  to  incur  the  burden  of  bid 
protests  to  govern  public  procurement  are:  (1 )  that  government  has  enormous  power  to  influence  the 
economy  and  the  fortunes  of  individual  firms  through  its  purchasing  decisions,  and  (2)  that  because 
the  expenditure  of  public  funds  is  at  issue,  and  the  government  should  take  extraordinary  precautions 
to  ensure  taxpayer  interests  are  protected. 

59  Selected  examples  of  the  literature  include  Grossman  and  Hart  (1983),  Sappington  (1991), 
Holmstom  and  Milgrom  (1991),  and  Prendergast  (1999). 

60  A  classic  solution  to  a  principal-agent  problem  is  found  in  the  popular  saying  “If  you  want  something 
done  right,  then  do  it  yourself.”  Principal-agent  problems  can  arise  in  a  number  of  different  ways.  One 
example  of  Conflicting  Objectives  is  a  situation  in  which  a  procurement  official  (PO)  might  bias  an 
award  to  a  favored  bidder  to  minimize  their  own  transaction  costs  (i.e.,  administrative  costs),  even 
though  that  bidder  offers  higher  production  (direct)  costs.  For  example,  it  is  often  easier  for  a  PO  to 
select  a  product  or  an  incumbent  firm  with  which  they  are  more  familiar.  (“No  one  was  ever  fired  for 
buying  from  IBM”).  A  second  example  of  Conflicting  Objectives  is  a  situation  in  which  there  are  issues 
related  to  asymmetric  information.  POs  may  not  be  as  well  informed  about  the  product  they  are 
buying  if  government  personnel  have  less  experience  than  their  private-sector  counterparts  and  if 
purchases  involve  idiosyncratic  (complex,  uncertain,  highly  differentiated)  goods  that  are  infrequently 
purchased.  A  key  question  revolves  around  incentives  for  POs  (agents)  to  educate  themselves  and 
negotiate  the  best  deal  for  the  government  (principal). 
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purchasing  agents  lack  the  same  incentives  as  their  private  sector  counterparts  to 
make  procurement  choices  that  maximize  interests  of  their  principals.  ...  Private 
firms  ...  develop  monitoring  and  incentive  schemes  that  [motivate]  purchasing 
agents  to  make  efficient  contracting  choices”  (Kovacic,  1995,  p.  467). 

The  principal-agent  problem  is  not  as  pronounced  when  standard 
(homogeneous)  products  like  food  products  are  being  procured.  In  this  case,  simply 
requiring  competitive  bidding  effectively  combats  incentive  problems.61  The 
procurement  official  has  little  discretion  and  any  attempt  to  award  a  contract  to  a  firm 
other  than  the  lowest  bidder  can  easily  be  detected. 

However,  the  focus  of  this  study  is  on  highly  differentiated  (heterogeneous), 
complex  products  (or  services)  that  allow  procurement  officials  to  retain  substantial 
discretion  in  formulating  and  applying  scoring  methodologies  to  evaluate  and  select 
alternatives  and  to  determine  the  optimal  exclusion  of  firms.  This  requires  highly 
competent  procurement  officials  who  are  presented  with  incentives  aligned  to  obtain 
the  best  possible  outcomes  for  our  troops  and  taxpayers.  Unfortunately,  institutional 
constraints  on  pay  and  benefits  limit  the  power  of  incentive  contracts  to  align  the 
interests  of  procurement  officials  closely  with  those  of  taxpayers  and  troops.  62 

Related  to  this  are  two  other  institutional  constraints,  rapid  turnover  and  the 
revolving  door.  Both  result  in  agents  (procurement  officials)  having  relatively  higher 
discount  rates,  i.e. ,  emphasizing  visible  current  benefits  of  a  procurement  action  that 
will  benefit  their  career  and  deferring  less  visible  costs  that  might  be  incurred  as  a 
result  of  that  procurement  action  until  after  they  depart.  This  is  especially  true  if  a 
procurement  official  views  his  or  her  current  employment  as  a  stepping  stone  either 


61  Competitive  procurements  also  help  overcome  the  challenge  of  asymmetric  information  (i.e.,  the 
relative  ignorance  of  buyers)  if  well-informed  sellers  compete  against  one  another  for  the  contract. 


62  With  a  highly  differentiated,  complex,  uncertain,  heterogeneous  product  (i.e.,  MDAP),  it  is  more 
difficult  to  verify  that  the  awardee  necessarily  offers  the  best  combination  of  price  and  quality,  hence 
the  need  for  more  sophisticated  oversight. 
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to  a  promotion  to  another  position  within  the  government  or  to  a  better  job 
opportunity  in  the  private  sector.63 

Compounding  these  incentive  problems  is  the  fact  that  agency  errors  are 
more  likely  to  occur  when  the  government  has  difficulty  retaining  personnel  because 
salaries  are  low  compared  to  the  private  sector  and  when  personnel  are  rapidly 
rotated  through  procurements  (with  the  corresponding  challenge  of  preserving 
accountability).  The  result  is  inadequate  experience  and  technical  knowledge,64 
which  requires  that  agents  depend  on  vendors  to  help  draft  solicitations,  creating  a 
further  wedge  between  the  principals’  and  the  agents’  objectives.  Of  course,  this 
also  creates  unfair  opportunities  for  some  firms  to  gain  an  advantage  in  an 
acquisition.  Third-party  oversight  of  the  procurement  process  is  needed  if  the 
government’s  objectives  are  not  equally  shared  by  procurement  personnel  who 
administer  the  process,  or  if  legitimate  errors  arise  that  can  bias  source  selection. 

Closely  related  to  this  is  the  “appropriability”  problem.  The  appropriability 
problem  is  an  incentive  problem  that  arises  when  procurement  officials  (agents) 
cannot  appropriate  (or  are  not  fully  compensated  for)  the  value  of  any  extra  effort 
they  make  in  their  positions,  and  it  arises  when  those  efforts  are  not  easily  measured 
or  monitored.  For  example,  including  extra  bidders  in  an  acquisition  can  potentially 
reduce  costs  for  the  government  (principal),  but  it  means  extra  work  for  a 
procurement  official  (agent).  An  appropriability  problem  exists  if  the  agent  incurs  the 
full  costs  (responsibility)  of  running  a  procurement  (evaluating  extra  bids,  etc.),  but 
receives  little  of  the  extra  benefits.  In  this  case,  for  example,  a  decision  to  bundle 


63  Other  institutional  constraints  such  as  yearly  budget  appropriations  make  planning  horizons  shrink, 
constraining  administrative  resources  and  resulting  in  end-of-year  use-it-or-lose-it  decisions,  which 
are  combined  with  reprogramming  constraints  that  govern  the  inter-temporal  substitution  of  funds. 
The  result  can  be  that  short-handed  personnel  are  asked  to  complete  procurement-related  tasks  in 
relatively  brief  periods,  where  the  measures  of  a  job  well  done  are  the  costs  incurred  in  running  the 
procurement  and  the  speed  with  which  the  final  product  is  delivered.  Suboptimization  at  different 
levels  in  the  organization  can  inadvertently  result  in  misaligned  incentives. 

64  This  is  well  documented  by  the  JAT  study  team  (2009)  as  a  “potential  root  cause”  of  successful 
protests. 
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several  contracts  together  might  actually  reduce  the  agent’s  costs  (and  the  costs  of 
the  transaction),  but  could  also  reduce  the  number  of  bidders,  thereby  limiting 
competition,  resulting  in  a  less  favorable  procurement  outcome. 

The  appropriability  problem  can  also  be  manifested  in  reduced  incentives  to 
test  products  or  in  overlooking  alternative  solutions  to  satisfy  demands,  say  by 
ignoring  new  products  or  vendors.  Of  course,  another  explanation  for  too  little  effort 
being  exerted  is  that  agents  (program  managers,  etc.)  are  not  allocated  sufficient 
resources.  In  this  case,  lower  effort  is  the  result  of  dedicated  program  managers 
simply  not  being  allocated  the  resources  to  accomplish  the  high-quality  results  that 
our  troops  and  taxpayers  deserve.65 

A  risk  acknowledged  both  in  the  literature  and  in  practice  is  that  conflicting 
objectives  of  procurement  officials  are  occasionally  reflected  in  fraud  and  favoritism 
(corruption),  or  in  inadequate  investments  in  human  capital  (errors).66  In  the  case  of 
corruption,  the  risk  is  that  procurement  officials  might  steer  an  award  to  a  favored 
bidder  in  exchange  for  private  benefits  (favors  or  bribes)  at  the  expense  of  higher 
public  costs.  Favoritism  can  be  generated  in  many  ways,  some  pernicious  (bid¬ 
rigging,  bribes,  expectations  of  future  employment,  etc.)  and  some  benign 


65  Alternatively,  too  little  effort  might  simply  be  a  manifestation  of  the  risk  of  over-deterrence, 
discussed  earlier.  Procurement  officials  risk  becoming  paralyzed  by  their  attempts  to  follow  the 
mountains  of  rules  and  regulations  that  currently  govern  the  procurement  process  and  that  are 
subject  to  oversight  by  the  protest  process. 

66  A  related  issue  is  the  incentive  problem  arising  from  conflicting  lines  of  authority — for  example,  a 
contracting  officer  and  a  program  (technical  or  project)  manager.  The  technical  officer  is  responsible 
for  assessing  agency  needs  and  writing  the  technical  portion  of  the  solicitation  and  for  assisting  in 
negotiations  and  technical  evaluation  of  proposals.  The  contracting  officer  has  primary  responsibility 
for  negotiations,  evaluation,  award,  and  administration  of  a  contract.  Technical  officers  are  motivated 
to  purchase  the  best  technology  available,  given  the  budget,  while  contracting  officers  want  to  assure 
regulatory  compliance  and  rapid  completion.  Technical  people  tend  to  support  higher  technology 
while  contracting  people  push  for  more  competition.  The  resulting  specifications  and  evaluation 
criteria  emerge  from  a  bargain  between  the  two,  including  issues  such  as  how  much  weight  to  give 
cost  versus  quality  (technical  factors/attributes)  in  the  scoring  function.  For  example,  “In  DoD,  there  is 
a  mind-set  that,  since  human  lives  are  at  stake  in  military  conflicts,  any  expense  is  justified  in 
acquiring  the  latest  and  best  technology”  (MMR,  1991,  p.  5).  We  abstract  from  this  potential 
complication  of  the  principal-agent  problem  by  aggregating  the  two  into  a  single  “procurement 
official.” 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  -  42  - 

NAVAL  POSTGRADUATE  SCHOOL 


(friendships,  personal  preferences,  etc.).  When  appropriability  problems  prevent 
government  procurement  officials  from  receiving  adequate  rewards  for  decisions  that 
benefit  taxpayers,  the  risk  is  that  “lower  pay  scales  and  weaknesses  in  the  incentive 
systems  used  to  motivate  public  employees  [will  make]  government  officials  ...  more 
prone  to  succumb  to  ...  temptations”  (Kovacic,  1995,  p.  488). 

These  “temptations”  can  be  reflected  in  abuses  of  discretion  by  procurement 
officials  such  as  (a)  failure  to  make  appropriate  quality-price  tradeoffs,  (b)  the 
tendency  to  skew  specifications  and  source  selection  decisions  to  favor  incumbent 
suppliers,  and  (c)  underinvestment  in  activities  that  maximize  taxpayer  interests.  In 
this  case,  third-party  oversight  mechanisms  help  ensure  that  procurement  officials 
follow  rules  and  regulations  and  do  not  unduly  restrict  competition.  Protests  can  be 
interpreted  as  a  powerful  oversight  tool  that  compensates  for  weaknesses  in 
incentives67  generated  by  conflicting  objectives:  “A  robust  protest  system  helps  cure 
principal-agent  problems  that  arise  in  the  public  procurement  system  ...  by 
increasing  the  likelihood  that  deviations  by  the  agent  (the  purchasing  agency)  from 
the  guidance  of  the  principal  (Congress)  will  be  detected  and  corrected”  (Kovacic, 
1995,  p.  486). 

Another  rationale  occasionally  heard  to  justify  third-party  monitoring  is  that 
government  purchasing  officials  are  less  competent  (more  prone  to  making  errors) 
than  private-sector  purchasing  agents:  “Compared  to  its  industry  counterparts,  [the 
DoD  contracting]  workforce  is  undertrained,  underpaid,  and  inexperienced”  (Kovacic, 
1995,  p.  487). 


67  The  intent  of  the  Congress  expressed  in  the  CICA  was  for  the  protest  process  to  encourage  private 
enforcement  of  laws  and  regulations  through  full  and  open  competition.  There  is  another  principal- 
agent  problem  in  enforcing  procurement  regulations  that  can  be  resolved  with  protests.  Rather  than 
depend  on  government  agencies  (auditors,  inspectors,  etc.)  to  monitor  compliance  in  the  public 
interest,  the  protest  process  aligns  the  private  interests  of  the  bidders/offerors  who  have  a  stake  in 
the  procurement  outcome  to  ensure  the  proper  conduct  of  the  procurement  (delegating  enforcement 
to  disappointed  bidders).  If  the  protest  machinery  provides  powerful  remedies  (stays,  the  award  of 
attorneys  fees,  re-competition,  etc.),  then  this  can  supply  strong  incentives  for  private  contractors  to 
provide  oversight  and  police  and  monitor  acquisitions. 
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The  protest  process  can  also  be  viewed  as  compensating  for  deficiencies  in 
the  skills  of  government  purchasing  personnel  by  reducing  their  discretion  and 
subjecting  their  decisions  to  more  oversight68:  “Protest  controls  are  one  part  of  a 
large  body  of  regulatory  safeguards  that  are  deemed  necessary  to  deter  and  punish 
ineptitude,  sloth,  or  corruption  of  public  purchasing  officials”  (Kovacic,  1995,  p.  469). 

However,  it  is  not  clear  that  any  empirical  analysis  has  conclusively 
demonstrated  government  purchasing  officials  make  less-sound  procurement 
choices  than  their  private-sector  counterparts.69 

There  is  scant  empirical  evidence  forjudging  whether  public  purchasing 
officials  are  more  prone  to  “shirk”  in  maximizing  taxpayer  interests  than 
private  purchasing  officials  are  to  shirk  in  maximizing  shareholder  interests,  or 
what  net  effect  ...  protest  controls  have  had  on  procurement  outcomes. 
(Kovacic,  1995,  p.  491) 

Protests  are  one  means  of  deterring  and  correcting  principal-agent  and 
appropriability  (agency)  problems  in  public  procurement.  Another  approach  is  to 
improve  public-sector  pay  and  training  for  procurement  officials — or  human  capital 
investments.  Internal  audits,  external  audits,  investigations,  sanctions,  and  penalties 
offer  other  approaches — all  of  which  are  attempts  to  increase  transparency  and 
accountability  in  procurement. 

A  primary  purpose  of  oversight  is  to  control  the  discretion  of  government 
officials  who  make  contract  awards,  since  their  incentives  are  often  poorly 
aligned  with  the  interests  of  taxpayers.  ...  [Oversight  induces  procurement 
officials  ...  to  make  purchase  decisions  that  are  more  consistent  with  the 
objectives  of  the  government.  (Marshall  et  al. ,  1991,  p.  2) 


68  Private-sector  purchasing  organizations  achieve  competition  by  strictly  limiting  discretion  of  their 
purchasing  officials.  For  instance,  General  Motors  information  technology  purchasing  process 
provides  for  clear  definition  of  scope  of  work,  one-time  negotiation,  firm-fixed  price  contracts,  and  a 
change  management  process  subject  to  robust  discipline.  Federal  purchasing  officials  are  not 
similarly  constrained  (see  Johnson,  2003). 

69  Literature  suggests  that  private-sector  purchasers  make  better  procurement  decisions  because  of 
greater  emphasis  on  competition  and  greater  constraints  on  buyer  discretion  (see  generally,  Johnson 
2003). 
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A  challenge  for  the  DoD  is  to  choose  an  optimum  mix  (portfolio)  of 
governance  mechanisms  that  will  minimize  the  costs  of  aligning  incentives — to 
reduce  the  risk  of  fraud  and  errors  and  to  encourage  competition — and  that  will 
guarantee  the  best  possible  procurement  outcomes  for  our  troops  and  taxpayers.70 

E.  Alternative  Governance  Mechanisms:  Human  Capital 
Investments  and  Third-Party  Monitoring  to  Address 
Principal-Agent  Problems  in  Procurement 

The  goal  of  military  procurements  is  to  obtain  the  best  (troop)  value  for 
(taxpayer)  money.  A  recent  study  suggests  this  can  be  accomplished  by  carefully 
adopting  one  of  six  ways  to  structure  an  acquisition.71  The  first  so-called  Economic 
Evaluation  of  Alternatives  (EEoA)  was  the  approach  originally  employed  by  RAND  to 
structure  the  KC-X  aerial  tanker  refueler  competition:  given  a  particular  user- 
determined  performance  and  schedule  combination,  select  the  vendor  that  offers  the 
lowest  (discounted)  total  system  life-cycle  costs.72  “Here,  an  ‘alternative’  can  be  a 
fleet  consisting  of  a  single  type  of  aircraft.  ...  [I]n  this  AoA,  the  most  ‘cost-effective’ 
alternative  means  precisely  the  alternative  whose  effectiveness  meets  the  aerial 
refueling  requirement  at  the  lowest  cost”  (Kennedy  et  al.,  2006,  p.  7). 

Structuring  procurements  and  evaluating  and  selecting  vendors  to  obtain  the 
greatest  cost  effectiveness  (or  value  for  money)  requires  a  delicate  balance  between 
rules  and  discretion.  This  is  an  important  issue  in  the  design  of  any  regulatory 


70  As  shown  in  the  companion  report,  DoD  must  also  ensure  that  its  agency  procurement  officials 
manage  bid  protests  in  ways  that  are  not  encumbered  by  parochial  or  personal  considerations  and 
that  align  with  DoD  interest  in  reducing  protest-related  costs  and  delays  (see  Benishek  et  al.,  (2011, 
pp.  107-113). 

71  The  main  contribution  of  these  six  techniques  is  to  offer  alternative  approaches  to  the  contentious 
issue  of  weighting  costs  and  performance — responsible  either  directly  or  indirectly  for  multiple 
protests  (see  Melese,  2009). 

72  The  second  EEoA  approach  is  for  procurement  officials  to  estimate  a  multi-year  budget  for  the 
overall  program,  and  then  to  select  the  vendor  that  offers  the  best  possible  performance  and 
schedule  combination  over  the  relevant  time  period.  Similar  to  these  first  two  approaches,  the 
remaining  four  require  procurement  officials  to  estimate  future  budgets  for  a  program,  in  addition  to 
estimating  the  costs  and  performance  of  individual  alternatives. 
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regime,  and  it  is  especially  relevant  in  attempts  to  re-shape  the  bid  protest  process. 
This  trade-off  between  rules  and  discretion  is  reflected  in  the  following  observation: 

When  we  design  organizations  based  on  rules,  we  guard  against  disaster,  but 
at  the  cost  of  stifling  excellence.  ...  Government  officials  deprived  of 
discretion  which  could  produce  misbehavior,  are  at  the  same  time  deprived  of 
discretion  that  could  call  forth  outstanding  achievement.  (Kelman,  1990,  p. 

28) 

As  discussed  earlier  in  the  context  of  over-deterrence,  a  heavily  rule-based 
approach  involves  significant  transaction  costs  that  can  adversely  affect  the 
efficiency  and  effectiveness  of  procurements,  but  on  the  other  hand,  greater 
discretion  exacerbates  the  principal-agent  problem.  To  complement  any  increased 
role  of  discretion,73  Arrowsmith  et  al.  (2000)  offer  a  valuable  reminder: 

Where  the  role  of  discretion  is  increased  in  the  procurement  system,  careful 
attention  needs  to  be  given  to  matters  such  as  professional  training  and 
recruiting  high-caliber  procurement  personnel — which  may  involve  ... 
enhancing  pay  or  the  career  development  structure,  or  recruiting  from  the 
private  sector,  (p.  85) 

The  appropriate  balance  between  rules  and  discretion  also  depends  on 
attitudes  towards  corruption  and  the  skills  of  procurement  personnel.  The  greater  the 
investment  in  human  capital  and  the  better  aligned  the  incentives,  the  greater  the 
opportunity  to  relax  regulatory  constraints. 

According  to  the  United  Nations  Guide  to  Enactment  of  the  Model  Law  on 
Procurement,  “an  effective  means  to  review  acts  and  decisions  of  the  procuring 
entity  and  procurement  procedures  ...  is  essential  to  ensure  the  proper  functioning 
of  the  procurement  system  and  to  promote  confidence  in  the  system”  (UNCITRAL, 
1995,  p.  99). 


73  “Much  greater  freedom  should  be  given  in  decisions  that  affect  a  manager’s  ability  to  accomplish 
his  mission.  Then  the  manager  should  be  held  responsible  for  how  well  he  does  in  accomplishing  the 
mission.  ...  [Ejnergy  that  now  goes  into  the  development  of  rules  to  restrict  public  officials  should  be 
redirected  toward  thinking  about  how  to  develop  resuits-based  performance  evaluation”  (Arrowsmith 
et  al.,  2000,  p.  78). 
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The  challenge  is  to  choose  the  most  cost-effective  portfolio  of  initiatives  that 
aligns  incentives  and  that  provides  adequate  monitoring  and  enforcement  of 
procurements  in  order  to  reduce  the  risk  of  fraud  and  errors  and  to  encourage 
competition.  In  searching  for  alternative  approaches,  the  optimum  mix  will  include 
targeted  investments  in  human  capital  as  well  as  centralized  (audits  and 
investigations)  and  decentralized  (bid  protests)  third-party  governance  mechanisms. 

Valuable  human  capital  investments  include  initiatives  for  building  integrity 
(e.g.,  ethics  training,  clearly  stated  and  widely  distributed  standards  of  conduct,  etc.), 
acquisition  training  (e.g.,  augmenting  skills  and  experience),  and  pay  for 
performance  (or  other  ways  to  align  incentives  to  improve  procurement  outcomes).74 
In  general,  third-party  governance  mechanisms  increase  transparency  and  improve 
accountability.  “Procurement  laws  which  emphasize  transparency  generally  contain 
extensive  provisions  to  [verify]  compliance  ...  [and]  some  mechanism  for  supervision 
and  enforcement,  [emphasis  added]  whether  by  the  administrative  hierarchy,  the 
public  or  legislature,  or  by  aggrieved  firms  through  a  complaint  [bid  protest] 
procedure”  (Arrowsmith  et  al.,  2000,  p.  75). 

Bid  protest  systems  are  an  example  of  a  decentralized  third  party  governance 
mechanism  that  relies  on  private  enforcement  to  increase  transparency  and 
accountability  in  procurement.75 


74  “Many  systems,  especially  those  which  emphasize  ...  transparency,  make  provisions  for  some 
means  of  external  enforcement.  ...  Of  course  the  extent  of  compliance  with  the  rules  will  also  be 
greatly  affected  by  ...  factors  such  as  the  quality  and  training  of  procurement  and  management 
personnel,  levels  of  pay  and  incentives,  and  the  general  ‘culture’  of  the  procuring  institution  [including] 
requirements  for  review  and  approval  of  procurement  decisions”  (Arrowsmith  et  al.,  2000,  p.  749). 


75  The  bid  protest  process  provides  decentralized  oversight  by  making  procurement  decisions  subject 
to  challenge  by  bidders  (or  offerors).  The  basic  premise  underlying  the  bid  protest  system  is  that  a 
robust  protest  mechanism  improves  procurement  performance  by  generating  competition  for 
government  contracts  and  by  monitoring  the  performance  of  government  officials  who  may  not 
exercise  discretion  to  the  benefit  of  taxpayers.  A  recommendation  is  to  collect  data  to  determine 
whether  the  protest  process  and  process  reforms  have  improved  the  performance  of  government 
buyers  and  procurement  outcomes. 
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In  order  for  a  procurement  process  to  achieve  value  for  money,  the  three 
most  frequently  cited  policy  goals  are  equity,  efficiency,  and  integrity: 

The  history  of  federal  procurement  law  demonstrates  a  concern  for  three 
policy  goals,  namely:  equity  (in  the  sense  of  fair  access  of  bidders  to  the 
procurement);  integrity  ( i.e.,  no  corruption);  and  efficiency  (meaning  the 
selection  of  [the  best  vendor],  ...  [A]  court  commenting  on  procurement  law 
noted  that  “the  purpose  of  these  statutes  ...  is  to  give  all  persons  equal  right 
to  compete  for  Government  Contracts;  to  prevent  unjust  favoritism,  or 
collusion  or  fraud  [in  awarding  government  contracts];  and  thus  to  secure  for 
the  Government  the  benefits  which  arise  from  competition.”  (as  cited  in 
Marshall  et  al. ,  1991,  p.  2) 

Protests  offer  a  type  of  third-party  oversight  mechanism  that  addresses 
principal-agent  problems  related  to  these  three  core  policy  goals:  integrity 
(minimizing  fraud  and  errors),  equity  (maximizing  competition),  and  efficiency  (value- 
for-money).  The  economics  approach  reveals  significant  (opportunity  and 
transaction)  costs  in  the  protest  process.  However,  we  must  be  careful  what  we  wish 
for.  It  is  important  to  recognize  that  any  modifications  of  the  protest  process,  or 
alternative  actions  and  initiatives  that  provide  oversight  of  the  procurement  process 
with  the  goal  of  promoting  integrity,  equity,  and  efficiency,  will  also  entail  significant 
(opportunity  and  transaction)  costs  as  well  as  risks  of  unintended  consequences.76 

The  operation  of  the  current  protest  system  reflects  an  assumption  that  the 
costs  of  protests  are  less  than  the  costs  of  relying  on  alternative  monitoring  and 
enforcement  mechanisms  to  achieve  the  same  policy  goals  (integrity,  equity, 
efficiency)  and  to:  “ensure  the  proper  functioning  of  the  procurement  system,  and  to 
promote  confidence  in  the  system”77  (Arrowsmith  et  al.,  2000,  p.  83).  A 


76  It  is  important  to  include  transaction  cost  considerations  in  the  economic  analysis  of  alternatives 
and  source  selection  decision  (see  Melese  et  al.,  2007).  These  include  the  administrative  costs  of  the 
award  process  and  the  costs  of  the  monitoring  and  oversight  of  protests,  audits,  and  other 
governance  mechanisms  to  promote  the  legal  integrity  and  economic  efficiency  of  the  procurement 
process  in  order  to  select  the  best  possible  products  and  services  on  behalf  of  troops  and  taxpayers. 

77  “Any  loosening  of  the  procurement  regulatory  straightjacket  should  be  accompanied  by,  and  linked 
to,  increased  resources  for  public  corruption  investigations.  ...  Deregulation  of  the  procurement 
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recommendation  is  to  evaluate  the  costs  and  benefits  of  alternative  approaches  that 
might  improve  procurement  outcomes — including  human  capital  investments  and 
third-party  governance  mechanisms.  The  full  costs  of  protests  must  include  the 
visible  costs  of  actual  protests  as  well  as  the  largely  hidden  costs  of  defensive 
practices  (Buy-offs,  Fed-mail,  and  over-deterrence)  to  avoid  protests. 

Table  1  offers  a  proposed  framework  to  collect  data  and  expert  opinions  to 
help  evaluate  alternative  governance  mechanisms.  The  first  column  lists  the 
evaluation  criteria,  including  the  procurement  policy  goals  of  integrity,  equity,  and 
efficiency.  To  achieve  these  policy  goals,  a  brief  list  (portfolio)  of  possible  alternative 
governance  mechanisms  appears  across  the  top. 

Bid  protests,  whistle-blower  lawsuits,  and  integrity  pacts  are  examples  of 
decentralized  third-party  governance  mechanisms.  Internal  audits,  external  audits, 
alternative  dispute  resolution  (ADR),  and  IG  investigations  are  examples  of 
centralized  oversight  mechanisms.  Centralized  and  decentralized  oversight 
mechanisms  can  act  either  as  complements  or  substitutes.  Carefully  crafted  human 
capital  investments  (training,  incentive  schemes,  etc.)  can  also  substitute  for  more 
costly  centralized  and  decentralized  oversight.78 


system  should  also  be  accompanied  by  an  increase  in  criminal  penalties  for  procurement  corruption” 
(Arrowsmith  et  al.,  2000,  p.  83). 

78  For  example,  major  investments  in  training  and  recruiting  experienced  procurement  officials  and  in 
establishing  new  policies  to  reward  successful  acquisition  outcomes  could  be  matched  with  a  change 
at  the  GAO  to  adopt  the  Court  of  Federal  Claims  (COFC)  default  of  innocent  until  proven  guilty  in  not 
granting  automatic  stays  that  disrupt  the  procurement  process  and  inadvertently  provide  Fed-mail  and 
Buy-off  incentives. 
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Table  1.  Evaluation  of  Alternative  Governance  Mechanisms:  Human  Capital 
Investments  and  Third-Party  Monitoring  to  Address  Principal-Agent  Problems 

in  Procurement 


Goal: 

VALUE-for-MONEY 

Protests 

1)  Agency 

2)  GAO 

3)  COFC 

Internal 

Audit; 

IG;  Peer 
Reviews 

External 

Audit; 

GAO; 

CAPE 

Inspector 

General;  ADR; 
Trusted  Agent; 
Integrity  Pacts 

Invest  in 

Human 

Capital 

Integrity. 

Minimize  Fraud  & 

Errors  (Transparency 
&  Accountability:  Anti- 
Corruption  & 

Regulatory 

Compliance) 

Green 

0 

Equity. 

Maximize  Competition 
(Increase  Participation) 

Yellow 

0 

Efficiency. 

Cost-effective 

Products  &  Services 
(Cost,  Performance  & 
Schedule) 

Red 

0 

BENEFITS 

(Advantages) 

COSTS 

(Disadvantages) 

The  stated  goal  of  this  project  is  to  offer  senior  decision-makers  a  useful 
framework  and  perspective  for  evaluating,  articulating,  and  supporting  protest  policy 
recommendations  that  improve  procurement  outcomes.  A  valuable  exercise  would 
be  to  refine  and  extend  the  list  of  alternative  approaches  that  appears  in  Table  1, 
identifying  actions  and  initiatives  that  substitute  for  (and  complement)  protests. 
Combining  data  with  best  practices  and  expert  opinion  to  fill  the  cells  of  the  matrix 
would  reveal  advantages  and  disadvantages  of  alternative  mixes  (portfolios)  under 
investigation.  The  results  in  each  cell  could  be  interpreted  through  a  stoplight  chart, 
such  as  that  illustrated  in  the  second  column  of  Table  1 .  The  last  two  rows  of 
cumulative  benefits  and  costs  could  then  help  guide  senior  leadership  in  reviewing 
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any  recommended  changes  in  the  protest  process  to  improve  procurement 
outcomes.79 

Completing  a  table  like  the  one  illustrated  in  Table  1  requires  evaluating  the 
benefits  (advantages)  and  costs  (disadvantages)  of  the  different  alternatives.80 
Traditional  centralized  oversight  mechanisms  rely  on  government  officials  to  allocate 
enforcement  resources.  This  generally  involves  an  auditor  or  inspector  general  who 
is  independent  of  the  regulated  activity  deciding  which  (pre-  or  post-award) 
procurements  to  review.  This  study  focuses  on  the  benefits  and  costs  of  the  bid 
protest  system,  a  decentralized  governance  mechanism  in  which  the  allocation  of 
enforcement  resources  is  mostly  decided  by  self-interested  contractors. 


79  The  matrix  could  also  help  evaluate  any  offsetting  (or  complementary)  actions  or  initiatives  that 
might  be  required  to  implement  proposed  changes  in  the  bid  protest  process  in  order  to  reduce 
overall  costs  and  delays. 

80 

Whereas  protests  tend  to  be  input  focused,  audits  and  investigations  tend  to  be  more 
output/outcome  focused — although  they  can  also  review  the  input  phase.  Given  sufficient  resources 
(increasing  transparency,  the  probability  of  a  review)  and  enforcement  power  (improving 
accountability,  the  consequences  of  a  good  or  bad  review),  centralized  audits  and  investigations  can 
have  a  powerful  impact  to  reduce  fraud  and  errors  and  increase  competition.  Since  they  are  initiated 
by  government  officials,  centralized  approaches  also  tend  to  avoid  many  of  the  pitfalls  of  protests 
such  as  Fed-mail,  Buy-offs,  and  over-deterrence — which  means  the  full  costs  of  conducting  these  are 
more  transparent.  A  disadvantage  of  audits  is  that  enforcement  is  often  weak  or  non-existent,  except 
perhaps  when  the  media  picks  up  the  story.  Also,  random  audits  are  likely  to  review  properly 
conducted  procurements,  while  protests  only  tend  to  occur  when  there  is  a  suspicion  of  impropriety. 
Two  further  observations  are  helpful:  (1)  the  auditor  may  be  poorly  informed  relative  to  a  disappointed 
bidder,  and  (2)  auditors  generally  lack  the  profit  motive  that  drives  the  behavior  of  a  protester  (private 
attorney  general).  The  decentralized  enforcement  of  disappointed  bidders  offers  better  alignment  if 
self-interested  private  attorneys  general  have  better  information  than  auditors  about  deviations  from 
procurement  statutes  and  regulations.  Another  cost  is  the  opportunity  costs  of  auditors  and  their  staffs 
and  the  agency  that  is  undergoing  an  audit,  which  includes  costs  (and  opportunity  costs)  of  collecting 
and  analyzing  information.  An  internal  audit  relies  on  ex-post  auditing  by  entities  within  the  agency 
(such  as  the  IG,  the  competition  advocate,  or  DCAA).  Audit  authorities  could  recommend  rewards  or 
punishments  for  procurement  officials,  adjustments  in  acquisition  procedures  and  techniques,  or 
payments  of  damages  to  wrongfully  excluded  offerors.  An  external  audit  is  one  alternative;  it  relies  on 
ex-post  auditing  by  external  observers  (such  as  the  GAO’s  auditing  unit).  Audit  authorities  could 
recommend  rewards  or  punishments  for  procurement  officials,  adjustments  in  acquisition  procedures 
and  techniques,  or  payments  of  damages  to  wrongfully  excluded  offerors.  Other  approaches  include 
ADR  and  other  preventions  and  resolution  tools  addressed  in  the  Better  Acquisition  Management 
companion  report;  posting  a  bond,  bringing  in  an  auditor  to  bless  the  procurement,  using  a  trusted 
agent  (ombudsman),  and  using  Transparency  International’s  integrity  pacts. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  -  51  - 

NAVAL  POSTGRADUATE  SCHOOL 


F.  A  Brief  Review  of  the  U.S.  Federal  Bid  Protest  System 

In  the  United  States,  a  disappointed  (or  prospective)  bidder/offeror  has  three 
major  options  to  contest  the  award  of  a  federal  contract.  Protests  can  be  filed  with 
the  procuring  agency  (C.F.R.,  48  §  33. 1 03[c]),  with  the  Government  Accountability 
Office  (GAO)  (U.S.C.,  31  §  3551  et.  seq),  or  with  the  U.S.  Court  of  Federal  Claims 
(COFC)  (U.S.C.,  28  §  1491  [b]).  The  distribution  of  protests  filed  over  the  last  several 
years  at  the  GAO  and  the  COFC  is  illustrated  in  Tables  2  and  3.  A  weakness  noticed 
in  the  literature  is  that:  “Unfortunately,  no  comprehensive  statistics  exist  that 
summarize  the  number  of  protests  at  the  agency  level  or  the  success  rates  of  such 
protests”  (Schaengold  et  al.,  2009,  p.  4). 

A  recommendation  of  this  study  is  for  the  DoD  to  collect  data  that  summarizes 
agency-level  protests  (in  a  manner  similar  to  the  data  routinely  displayed  for  GAO 
protests  and  the  COFC).  Also,  because  most  agency  protest  decisions  are  not 
published,  there  is  limited  transparency,  and  the  lack  of  a  record  of  precedent 
deprives  future  protesters — and  the  agency  itself — of  the  benefit  of  the  agency’s 
reasoning  on  prior  decisions. 

The  number  of  bid  protests  filed  with  the  COFC  is  a  small  fraction  of  those 
filed  with  the  GAO.  For  example,  in  FY2007,  protests  filed  with  the  COFC  (70)  were 
less  than  5%  of  those  filed  with  the  GAO  (1 ,41 1 ).  As  a  consequence,  the  discussion 
below  focuses  on  GAO  protests. 

According  to  Worthington  and  Goldsman  (1998),  two  (possibly  conflicting) 
concerns  are  reflected  in  any  protest  process:  efficiency  and  effectiveness.  An 
“efficient”  bid  protest  process  ensures  that  protests  are  handled  expeditiously, 
thereby  minimizing  disruptions  to  the  acquisition  process.  An  “effective”  system 
(related  to  the  criteria  of  integrity  in  the  matrix  in  Table  1 )  ensures  that  disappointed 
bidders  are  given  a  forum  to  air  their  grievances  and  that  agency  officials  are  held 
accountable  for  their  actions  (Worthington  &  Goldsman,  1998,  p.  472). 
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Table  2.  GAO  Bid  Protest  Statistics,  FY2001-FY2008 

(Schaengold  et  al.,  2009,  p.  255) 


FY 

2001 

2002 

2003 

2004 

2005 

2006 

2007 

2008 

Protests  Filed 

1, 146 

1,204 

1,352 

1,485 

1,356 

1,327 

1,411 

1,652 

Protests  Closed 

1098 

1133 

1244 

1405 

1341 

1274 

1393 

1581 

Closed  Protests 

Resolved  on  their 

Merit 

311 

256 

290 

365 

306 

249 

335 

291 

Sustained  in 

Whole  or  Part 

66(21%) 

41(16%) 

50(17%) 

75(21%) 

71(23%) 

72(29%) 

91(27%) 

60(21%) 

Effectiveness  Rate 

33% 

33% 

33% 

34% 

37% 

39% 

38% 

42% 

Cases  Resolved 

Through  ADR 

150(84%) 

145(84%) 

120(92%) 

123(91%) 

103(91%) 

91(96%) 

62(85%) 

78(78%) 

GAO  Hearings  for 
Fully  Developed 

Decisions 

63(12%) 

23(5%) 

74(13%) 

56(9%) 

41(8%) 

51(11%) 

41(8%) 

32(6%) 

d 

Table  3.  Bid  Protests  at  the  U.S.  Court  of  Federal  Claims,  FY2000-FY2007 

(Schaengold  et  al.,  2009,  p.  255) 


FY 

2000 

2001 

2002 

2003 

2004 

2005 

2006 

2007 

Number  of  Cases 
Involving  Bid  Protests 

54 

57 

39 

55 

69 

61 

73 

70 

Percentage  of  Cases 
Involving  Bid  Protests 

6.8% 

7.3% 

2.6% 

1.8% 

2.9% 

3.6% 

6.8% 

6.4%„ 

Two  of  the  three  U.S.  protest  forums  are  described  in  detail  in  Appendix  1 
Agency-Level  Protests  and  the  GAO  Bid  Protest  Process.  Appendix  1  offers  a  more 
precise  and  detailed  overview  that  includes  protest  process  flowcharts  developed  for 
each  of  the  three  major  forums. 
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An  Economic  Framework  &  Potential 
Change  Candidates:  A  Decision  Model 


As  discussed  earlier,  the  objective  of  firms  doing  business  with  the 
government  is  to  maximize  profits,  i.e.,  to  provide  a  market-based  return  on  capital 
to  shareholders.  Meanwhile,  the  intent  of  a  protest  system  is  to  provide  a 
decentralized  governance  mechanism  to  oversee  the  integrity,  equity,  and  efficiency 
of  the  procurement  process.  Although  the  government’s  intent  is  for  protesters  to  act 
as  a  type  of  third-party  oversight  of  government  buyers  (procurement  officials,  etc.), 
the  reality  is  that  because  protesters  themselves  are  in  competition  as  sellers,  they 
have  conflicting  objectives.  Their  motivation  is  not  simply  to  detect  errors  and  fraud, 
but  to  maximize  profits. 

As  this  study  has  highlighted,  profit  maximization  can  lead  firms  to  undertake 
protests  for  reasons  substantially  different  from,  and  fundamentally  opposed  to,  the 
government’s  objectives  (i.e.,  to  minimize  errors  and  fraud  on  the  part  of 
procurement  officials  and  to  increase  competition).  Firms  use  the  protest  process 
strategically  to  improve  their  competitive  bargaining  position  and  are  likely  to  be 
equally  strategic  in  their  decision  of  whether  to  protest  an  award. 

The  dual  objective  of  this  section  is  to  present  an  economic  framework  of  a 
representative  bidder’s  decision  whether  to  (i)  participate/compete  in  a  public 
procurement  and  (ii)  protest  in  the  event  they  lose.  This  requires  converting  the 
legal/regulatory  protest  process  flowchart  (illustrated  in  Figure  4)  into  a 
representative  bidder’s  decision  tree  (similar  to  Figure  2).  This  exercise  reveals  a  set 
of  variables  under  government  control  that  can  influence  a  contractor’s  protest 
decision.  Among  these  variables  are  key  factors  that  impact  the  expected  benefits 
and  costs  to  a  disappointed  bidder  of  filing  a  protest. 
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A.  Highlights  of  the  GAO’s  Bid  Protest  System :  A  Protest 
Process  Flowchart 

Figure  4  offers  a  detailed  process  flow  of  actions  and  events  that  impact  three 
key  players:  a  protester,  the  agency  defending  against  the  protest,  and  the  GAO.  It 
is  useful  to  highlight  several  aspects  of  Figure  4. 81 

An  agency  has  5  options  with  which  to  respond  to  the  filing  of  a  protest.  The 
first  option  for  the  agency  is  to  move  to  dismiss  the  protest  on  the  grounds  that  the 
protest  was  frivolous,  meritless,  or  outside  the  GAO’s  jurisdiction.  This  option  allows 
the  agency  to  avoid  up  to  100  days  of  delay  (the  statutory  time  limit  for  a  GAO  bid 
protest);  to  preserve  the  agency’s  acquisition  strategy,  its  competitive  range,  or  its 
award  decision;  to  prevent  discovery  and  extensive  production  of  documents;  to 
save  the  agency’s  legal  and  administrative  costs  of  defending  the  protest;  and  to 
avoid  potential  payments  of  protester’s  legal  fees  as  well  as  protester’s  bid  and 
proposal  preparation  costs.  The  protester  can  respond  to  the  agency  motion  by 
withdrawing  the  protest  or  opposing  the  motion.  If  the  protester  opposes  the  motion, 
the  GAO  may  dismiss  the  protest  with  a  summary  dismissal,  dismiss  with  a 
dismissal  opinion,  or  refuse  to  dismiss  the  protest. 

The  agency’s  second  option  is  to  undertake  corrective  action  by  affording  the 
protester  some  form  of  relief.  This  option  allows  the  agency  to  avoid  up  to  100  days 
of  delay  (the  statutory  time  limit  for  a  GAO  bid  protest);  to  save  the  agency’s  legal 
and  administrative  costs  of  defending  the  protest;  and  to  avoid  potential  payments  of 
protester’s  legal  fees.  However,  corrective  action  may  involve  changing  or  reversing 
the  agency’s  acquisition  strategy,  its  competitive  range,  or  its  award  decision; 
payment  of  protester’s  bid  and  proposal  preparation  costs;  or  other  relief  to  the 
protester.  The  protester  may  either  agree  to  the  corrective  action  or  request  further 


81  This  process  description  is  further  described  in  Chapter  11(C),  Guide  to  Agency  Management  of  the 
Protest  Process  to  Minimize  Time  Delays  and  Costs  in  Benishek  et  al.  (2011,  pp.  10-41). 
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corrective  action.  An  agency  may  refuse  further  corrective  action  and  proceed  to  a 
third  or  fourth  option. 

The  agency’s  third  option  is  to  engage  in  alternative  dispute  resolution  (ADR). 
This  option  allows  the  agency  to  avoid  up  to  100  days  of  delay  (the  statutory  time 
limit  for  a  GAO  bid  protest);  to  save  the  agency’s  legal  and  administrative  costs  of 
defending  the  protest;  and  to  avoid  potential  payments  of  protester’s  legal  fees. 
However,  corrective  action  may  involve  changing  or  reversing  the  agency’s 
acquisition  strategy,  its  competitive  range,  or  its  award  decision;  payment  of 
protester’s  bid  and  proposal  preparation  costs;  or  other  relief  to  the  protester.  The 
protester  may  either  agree  to  use  ADR,  or  disagree  to  use  ADR.  If  the  protester 
agrees  to  use  ADR,  the  ADR  can  take  the  form  of  negotiation  assistance  facilitated 
by  the  GAO  or  ADR  without  GAO  involvement,  such  as  direct  negotiation,  mediation, 
or  arbitration.  If  ADR  succeeds,  it  can  result  in  corrective  action  or  withdrawal  of  a 
protest.  If  ADR  fails,  the  agency  can  proceed  to  option  4,  agency  defense  by 
agency  report.  The  regulatory  timing  for  filing  the  agency  report  within  30  days  from 
the  date  of  award  is  not  affected  by  the  agency’s  decision  to  pursue  ADR. 

The  agency’s  fourth  option  is  to  defend  the  procurement  by  filing  its  defense 
in  the  form  of  an  agency  report  within  30  days  of  the  filing  of  the  protest.  This  option 
allows  the  agency  an  opportunity  to  protect  its  acquisition  strategy,  competitive 
range  decision,  or  award  decision.  However,  the  agency  will  incur  a  delay  of  up  to 
100  days  (the  statutory  time  limit  for  a  GAO  bid  protest)  and  incur  legal  and 
administrative  costs  of  defending  against  a  protest.  Further,  if  an  agency  takes  this 
option  and  subsequently  loses  the  protest,  it  will  be  obligated  to  pay  the  protester’s 
legal  fees  and  costs.  If  an  agency  takes  this  option  and  subsequently  loses  a 
protest  after  a  “best  interests”  override  was  put  in  place,  the  law  authorizes  the  GAO 
to  recommend  relief  regardless  of  any  impact  on  the  program. 

A  protester  may  file  opposing  comments  or  withdraw  a  protest.  The  protester 
may  also  file  a  supplemental  protest.  The  agency  is  able  to  exercise  the  same 
response  option  in  regards  to  the  supplemental  protest  as  in  regards  to  the  original 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  -  57  - 

NAVAL  POSTGRADUATE  SCHOOL 


protest,  and  GAO  will  consolidate  all  protests  together  for  a  single  resolution.  The 
protester  may  also  request  a  hearing.  The  GAO  may  conduct  a  hearing  on  one  or 
all  protests,  in  which  case  the  parties  will  be  asked  to  file  post-hearing  comments. 

Following  the  full  exchange  of  comments,  the  GAO  may  conduct  outcome 
prediction  ADR.  During  outcome  prediction,  GAO  will  recommend  either  to  sustain 
or  to  deny  the  protest.  If  the  GAO  recommends  to  sustain  the  protest  and  the 
agency  agrees,  an  agency  may  take  corrective  action  in  accordance  with  GAO 
recommendation.  The  protest  is  then  dismissed  as  academic.  If  the  GAO 
recommends  a  denial  or  if  the  agency  or  the  protester  disagree  with  the  outcome 
prediction,  they  can  ask  GAO  to  issue  an  opinion. 

Once  the  GAO  issues  an  opinion  with  or  without  outcome  prediction,  the 
parties  may  ask  for  reconsideration.  If  the  GAO  opinion  (including  the 
reconsideration  request)  denies  the  protest,  the  protester  may  file  a  protest  at  the 
Court  of  Federal  Claims.  If  the  GAO  opinion  (including  reconsideration)  sustains  the 
protest  and  recommends  relief,  the  agency  can  provide  the  protester  the 
recommended  relief  or  refuse  to  follow  GAO’s  recommendation.  The  GAO  will 
report  the  agency  to  Congress  and  recommend  sanctions  or  relief  including  private 
relief  legislation,  rescission  or  cancellation  of  funds,  Congressional  investigation,  or 
other  action.  Further,  if  the  agency  refuses  to  follow  the  GAO’s  recommendation,  it 
can  request  the  Office  of  Federal  Procurement  Policy  within  the  Office  of 
Management  and  Budget,  or  the  Office  of  Legal  Counsel  within  the  Justice 
Department,  to  issue  an  opinion  supporting  its  position. 

In  addition  to  the  fourth  option,  the  agency  may  also  undertake  the  fifth 
option:  a  request  for  an  express  option.  This  request  must  be  made  not  later  than  5 
days  after  protest  filing.  Under  this  option,  the  agency  would  defend  the 
procurement  by  filing  its  defense  in  the  form  of  an  agency  report  within  30  days  of 
the  filing  of  the  protest.  This  option  allows  the  agency  an  opportunity  to  protect  its 
acquisition  strategy,  competitive  range  decision,  or  award  decision.  However,  the 
agency  will  incur  a  delay  of  up  to  65  days  instead  of  the  usual  1 00  days  (the 
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statutory  time  limit  for  a  GAO  bid  protest)  and  incur  legal  and  administrative  costs  of 
defending  against  a  protest.  Further,  if  an  agency  takes  this  option  and 
subsequently  loses  a  protest,  it  will  be  obligated  to  pay  a  protester’s  legal  fees  and 
costs.  If  an  agency  takes  this  option  and  subsequently  loses  a  protest  after  a  “best 
interests”  override  was  put  in  place,  the  law  authorizes  the  GAO  to  recommend  relief 
regardless  of  impact  on  the  program. 
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Figure  4.  Agency  Response  to  Bid  Protest  at  the  GAO 
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B.  The  Disappointed  Bidder’s  Problem:  A  Decision  Tree 

The  final  GAO  opinion,  issued  within  100  days  (or  65  days  under  the  express 
option),  will  either  sustain  or  deny  the  protest.  If  the  protest  is  sustained,  then  the 
protester  earns  an  award,  represented  by  Xa.  If  the  protest  is  dismissed  or  denied, 
then  we  assume  the  protester  settles  for  (or  pursues)  the  next-best  alternative 
project  whose  value  is  Xo. 

Recognizing  the  competing  interests  and  conflicting  objectives  of  the  different 
players  in  the  protest  process — i.e.  assuming  the  protester  maximizes  profits, 
including  future  business  opportunities;  the  agency  maximizes  troop  value  for 
taxpayer  money;  and  the  GAO  acts  as  an  arbiter — suggests  converting  the  protest 
process  flow  into  a  protester  decision  tree.  Streamlining  and  simplifying  the  protest 
process  illustrated  in  detail  in  Figure  4  suggests  a  straightforward  decision  tree 
similar  to  that  illustrated  in  Figure  2.  This  new  decision  tree  offers  a  slight  extension 
to  the  disappointed  bidder  decision  problem  developed  earlier. 

The  decision  tree  illustrated  in  Figure  5  underpins  the  economic  framework 
developed  in  this  section.  The  disappointed  bidder  must  decide  whether  to  file  or  not 
file  a  protest.  As  discussed  earlier,  a  typical  firm  will  calculate  the  expected  value 
(profit)  of  filing  a  protest,  i.e.,  its  expected  benefits  and  expected  costs.  It  will  only  file 
if  it  forecasts  greater  profits  from  protesting  than  from  not  protesting.  In  terms  of 
expected  benefits,  “[protesters  ...  care  about  two  ...  questions:  how  difficult  is  it  ... 
to  win,  and  does  winning  a  protest  bring  any  meaningful  relief”  (Gordon,  2006,  p. 

10).  The  latter  question  relates  the  magnitude  of  the  award  from  winning  a  protest, 
Xa.  The  first  question  is  slightly  more  involved  in  that  it  requires  two  steps. 

As  indicated  above,  in  order  to  be  successful,  a  protest  must  first  be 
determined  to  have  merit  (with  probability  Pm)  before  it  can  be  decided  by  the  GAO. 
Then,  given  that  it  has  merit,  it  must  be  sustained  (the  conditional  probability  Ps). 
“Where  the  protest  survives  initial  screening,  the  critical  question  ...  will  be  the 
standard  of  proof  that  must  be  met  [to]  sustain  the  protest”  (Gordon,  2006,  p.  10). 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY  -  61  - 

NAVAL  POSTGRADUATE  SCHOOL 


A  recent  RAND  study  discovered  that  a  total  of  836  protests  were  filed  with 
the  GAO  against  the  USAF  over  the  period  FY2000-FY2008.  Only  201  were  found 
to  have  merit  and  of  those,  29  were  sustained  (these  calculations  were  based  on 
root  B  number  counts  derived  from  PACTS,  Camm  et  al.,  2009,  p.  6).  Based  on  this 
USAF  data,  less  than  25%  of  protests  filed  were  found  to  have  merit.  The  probability 
(relative  frequency)  that  a  protest  has  merit  is  Pm  =  201/836  =  0.24  =  24%.  Of  those 
201  protests  found  to  have  merit,  less  than  15%  were  sustained.  The  conditional 
probability  that  a  protest  will  be  sustained  given  that  it  has  merit  is  Ps=  29/201  = 
0.144  =  14%.  These  probabilities  (relative  frequencies)  can  be  used  by  a 
representative  bidder  interested  in  competing  for  an  Air  Force  contract  simply  by 
inserting  them  into  a  decision  tree  like  the  one  in  Figure  5. 


Figure  5.  A  Disappointed  Bidder  Decision  Tree 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-62  - 


Data  for  the  period  FY200 1-2008  is  presented  in  Table  4  for  the  Department 
of  Defense  (DoD)  as  a  whole.  The  conditional  probability  that  a  protest  will  be 
sustained,  Ps,  appears  in  the  fifth  row  in  Figure  9  and  is  called  the  DoD  Sustain 
Rate. 

Table  4.  Protests  Filed  with  the  GAO  against  DoD  Procurements 

(JAT,  2009,  p.  9) 


DoD  Cases  Closed 

603 

680 

704 

731 

706 

739 

775 

838 

(B-  N  brs) 

(60%) 

(65%) 

(62%) 

(56%) 

(58%) 

(60%) 

(62%) 

(58%) 

DoD  Merit  (Sustain 
+  Deny  Decisions) 
(B-Nbrs) 

174 

157 

184 

213 

154 

151 

200 

153 

DoD  #  of  Sustains 
(B-Nbrs) 

40 

22 

32 

36 

28 

38 

62 

30 

DoD  #  of  Sustains 
(Procurements) 

N/A 

15 

19 

14 

17 

21 

19 

11 

DoD  Sustain  Rate 
(B-Nbrs) 

23% 

14% 

17% 

17% 

18% 

25% 

31% 

20% 

DoD  Competitive 
Actions 

296  946 

376  006 

460,528 

549  079 

1,174.786 

1.099,142 

1 .092.434 

1,176,913 

DoD  Competitive 

$8338 

$101 B 

$122.7B 

$138.6B 

$1683B 

$184.7B 

$205.48 

$251. IB 

Dollars 

(up  21%) 

(up  21%) 

(up  13%) 

(up  21%) 

(up  10%) 

(up  11%) 

(up  22%) 

■Percentages  shown  for  DoD  Cases  Closed  represent  DoD  percentage  of  the  total  GAO  cases  closed 
based  on  CRS  Report  data  (www.gao.gov) 

■DoD  Procurements  represent  actual  number  of  solicitations  associated  with  B  #s  as  reported  try  GAO 
■DoD  Sustain  Rate  calculated  by  dividing  DoD  sustained  B  #s  by  Merit  Decisions 
■N/A  =  Data  Not  Available 

■DoD  Competitive  Actions  and  Dollars  percentage  variations  are  only  in  relation  to  the  previous  year 
■Competitive  Actions/Dollars  does  not  include  DECA  (Source:  FPDS-NG) 


This  data  indicates  that  in  the  most  recent  year  of  the  sample  (FY2008),  for  all 
of  the  DoD,  Ps  =  0.20.  In  other  words,  only  20%  of  the  protests  found  to  have  merit 
were  subsequently  sustained.  An  investment  in  transparency  by  publishing  merit  and 
sustain  rates  might  pay  for  itself  through  a  reduction  in  the  rate  of  protests  if  it  serves 
as  a  deterrent  to  disappointed  bidders  who  might  otherwise  be  tempted  to  protest 
due  to  an  optimism  bias.82 


82  Note  that  if  companies  use  protests  as  a  strategic  bargaining/negotiating  tool  to  extract 
concessions  from  procurement  agencies  (Fed-mail  or  Buy-offs),  then  they  will  not  be  easily  deterred 
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C.  Expected  Benefits  of  Winning  a  Protest  and  Expected 
Costs  of  Losing 

There  is  only  one  way  a  disappointed  bidder  can  win  a  protest:  the  protest 
must  have  merit  and  be  sustained  given  that  it  has  merit.  In  calculating  its  expected 
benefits  from  filing  and  winning  a  protest,  a  contractor  multiplies  the  (tiny)  probability 
that  its  protest  is  sustained  (i.e.,  the  probability  that  the  protest  has  merit  times  the 
probability  that  it  is  sustained,  given  that  it  has  merit,  or  Pmx  Ps  =  24%  x  14.4%  = 
3.5%  =  29/836),  by  the  value  of  the  award,  Xa  for  example,  see  Figure  5). 

However,  deducted  from  the  award,  a  firm  must  subtract  its  overall  bidding 
costs  (Cb)  and  any  filing  fees  (Cf),  and  anticipate  whatever  costs  it  decides  to  invest 
to  conduct  the  protest  (/).  In  general,  the  expected  value  of  winning  a  protest  to  a 
firm  that  loses  the  competition  is  as  follows: 

Expected  Benefit  of  Winning  =  (Pmx  Ps)  x[Xa-  Cb  -  Cf-  f\  (1) 

While  there  is  only  one  way  the  disappointed  bidder  can  win  a  protest — the 
protest  must  have  merit  (Pm  -  0.24)  and  must  be  sustained  given  that  it  has  merit  (Ps 
=  0.144) — there  are  two  ways  it  can  lose  the  protest:  (i)  if  the  protest  is  dismissed, 
with  probability  (1  -  Pm)  =  0.76,  or  (ii)  if  the  protest  has  merit  but  is  subsequently 
denied,  perhaps  due  to  lack  of  prejudice,  with  probability  P/w(1  -  Ps)  =  0.76  x  0.86  = 
0.65.  Therefore,  a  disappointed  bidder’s  expected  cost  calculation  includes  two 
components,  or  two  ways  to  lose  a  protest. 

In  the  event  that  a  bidder  loses  the  protest,  the  value  of  the  next-best 
alternative  project  (or  agency  settlements)  is  given  by  Xo.  This  is  offset  by  the  usual 
suspects  (bidding  costs,  filing  costs,  and  any  protest-related  investments  the  firm 
lost  in  the  current  competition).  We  add  possible  penalties,  Cf,  in  the  event  the 
protest  is  found  to  be  without  merit  (e.g.,  dismissed  as  frivolous),  and  C2,  in  the 


by  evidence  that  the  large  majority  of  protests  are  found  to  be  without  merit,  and  that  the  majority  of 
those  with  merit  are  not  sustained. 
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event  the  protest  has  merit  but  is  ultimately  denied.83  In  general,  the  expected  value 
of  losing  the  protest  to  a  firm  that  loses  the  competition  is  given  by  the  following 
equation: 

Expected  Cost  of  Losing  =  (1  -  Pm)  [Xo  -  Cb-  Cf  -  I  -  Ci]  +  Pm(  1  -  Ps)[Xo  -  Cb-  Cf 
-I-C2]  (2) 


The  first  term  represents  the  protest  being  dismissed  and  the  second 
represents  the  protest  being  denied.  A  streamlined  version  of  a  representative 
disappointed  bidder’s  protest  decision  of  whether  to  file  a  protest  is  illustrated  in 
Figure  5. 

D.  Strategies  to  Reduce  Protests 

As  mentioned  in  Section  II,  reducing  protests  can  be  accomplished  in  two 
ways:  by  reducing  the  expected  benefits  of  winning  a  protest,  shown  in  Equation  1, 
or  by  increasing  the  expected  costs,  shown  in  Equation  2.  Equation  1  demonstrates 
that  reducing  the  expected  benefits  of  filing  a  protest  could  be  accomplished  by 
reducing  awards/payoffs  in  the  event  that  protests  are  sustained,  Xa,  or  by 
increasing  filing  fees  (and/or  other  related  expenses),  Cf.  Reducing  the  expected 
benefits  of  a  protest  could  also  be  accomplished  by  reducing  the  probability  that  a 
protest  has  merit,  Pm,  or  that  it  is  sustained  given  that  it  has  merit,  Ps. 

The  probability  a  protest  has  merit,  Pm,  is  likely  to  be  positively  correlated  with 
agency  errors,  E,  and  fraud,  F.  So,  if  Pm  =  Pm  (E,  F),  then  any  investment 
(efforts/initiatives)  to  reduce  fraud  or  errors  will  reduce  the  probability  a  protest  has 
merit  and  the  expected  benefits  of  a  protest. 

Similarly,  we  can  assume  that  the  probability  a  protest  is  sustained  given  that 
it  has  merit  (or  corrective  action  is  taken),  Ps,  is  positively  correlated  with  fraud  and 


83  There  is  precedent  for  quantifying  the  latter  cost  in  that  the  COFC  Rule  65  requires  a  disappointed 
bidder  to  post  a  bond  that  it  loses  in  the  event  that  its  protest  is  ultimately  denied. 
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errors  as  well  as  correlated  with  the  amount  of  protest-related  investments  made  by 
the  firm,  /,  and  the  quality  (technical)  measure  of  effectiveness  that  the  firm  offers  in 
its  bid  proposal,  MOE.  However,  it  is  assumed  this  probability  is  inversely  correlated 
with  the  contract  bid  price  offered  by  the  firm,  P.  Ceteris  paribus,  the  higher  the 
price  offered  by  the  firm  relative  to  its  competitors,  the  less  likely  a  protest  will  be 
sustained  given  that  it  has  merit,  such  that  Ps  =  Ps(E,F,l,MOE,P).84 

Since  I,  MOE,  and  P  are  all  contractor  decision  variables,  the  government’s 
attempt  to  reduce  the  probability  that  a  protest  is  sustained  given  that  it  has  merit  is 
once  again  left  with  investments  (efforts/initiatives)  to  reduce  fraud  or  errors. 

Equation  2  shows  that  increasing  the  expected  costs  could  be  accomplished 
with  similar  actions  and  initiatives,  but  it  also  includes  the  possibility  of  assessing 
penalties  (or  increasing  costs),  Ci  and  C2,  or  somehow  raising  the  value  of  the  next- 
best  alternative,  Xo  (possibly  by  unbundling  the  contract  or  with  shared 
procurements,  i.e.,  split  buys). 

E.  A  Representative  Bidder  Model 

The  protest  problem  just  discussed  is  actually  a  piece  of  a  bigger  profit 
maximization  problem,  that  any  representative  bidder/offeror  faces  upon  entering  a 
competition  for  a  government  contract.85  The  representative  bidder  model  described 
below  allows  for  a  more  detailed  examination  of  this  problem. 


84  It  seems  reasonable  to  assume  that  the  lower  the  quality  (MOE)  and  the  higher  the  price  (P)  a  firm 
offers,  the  less  likely  it  is  to  be  prejudiced  (or  the  lower  its  chance  of  winning  the  competition),  and, 
therefore,  the  lower  the  probability  that  the  GAO  will  sustain  a  protest  given  that  it  has  merit. 

85  Developing  the  broader  problem  could  offer  important  new  insights.  For  example,  suppose  the 
government  decided  to  raise  filing  fees  (Cf),  assess  penalties  if  a  protest  is  summarily  dismissed  as 
frivolous  (i.e.,  cost,  Cl),  or  impose  a  “loser  pays”  penalty  if  a  protest  is  ultimately  denied  (i.e.,  cost, 
C2)  in  order  to  raise  protest  costs  and  limit  the  number  of  protests  (i.e.,  demand  curves  slope  down). 
The  broader  representative  bidder  model  might  reveal  that  bid  prices,  P,  offered  by  representative 
bidders  would  tend  to  increase,  so  that  we  might  discover  that  some  of  the  cost  is  ultimately  shifted 
back  onto  the  government  as  higher  acquisition  costs. 
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Fortunately,  the  expected  benefits  of  winning  a  protest,  shown  in  Equation  1, 
and  the  expected  costs  of  losing  a  protest,  shown  in  Equation  2,  are  major 
components  of  the  representative  bidder’s  profit  function.  In  fact,  a  representative 
bidder’s  total  expected  profits,  V,  consist  of  only  two  additional  components.  The  first 
is  the  expected  value  of  winning  the  competition  and  the  second  is  the  expected 
value  of  losing  the  competition. 

The  probability  that  a  representative  bidder  wins  the  competition,  Pw,  is 
assumed  to  be  negatively  correlated  with  the  price  it  offers  (P),  any  agency  fraud  (F), 
or  errors  (E)  and  with  the  number  of  bidders  (A/),  but  positively  correlated  with  the 
quality  (technical)  measure  of  effectiveness  it  offers  ( MOE ),  such  that  Pw  = 
Pw(P,E,F,N,MOE).  Assuming  the  government  requires  a  fixed  quantity,  Q  (e.g.,  179 
KC-X  aerial  tanker  refuelers),  the  representative  bidder’s  problem  involves  choosing 
a  price  (P),  quality/schedule  (MOE),  and  how  much  to  invest  in  post-award,  protest- 
related  expenditures  (/)  in  order  to  maximize  its  total  profit  function 
( V ),  where 

V  =  the  expected  value  of  winning  the  competition, 

+  the  expected  value  of  losing  the  competition, 

+  the  expected  benefit  of  losing  the  competition  but  winning  the 

protest, 

+  the  expected  cost  of  losing  the  competition  and  losing  the  protest. 

Because  Equation  1  is  the  expression  for  winning  the  protest  and  Equation  2 
for  losing  the  protest,  there  are  only  two  missing  components  in  the  profit  function. 

The  first  is  the  expected  value  of  winning  the  competition,  say  E(W).  This  is 
simply  the  probability  of  winning  the  competition  (Pw)  multiplied  by  the  payoff  from 
winning.  The  payoff  includes  the  award  of  the  contract  (revenues  Px  Q),  minus 
production  costs  (total  costs  C(Q,MOE)),  minus  the  costs  of  participating  in  the  bid 
and  proposal  process  (Ce),  minus  any  expected  costs  of  having  to  defend  the 
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contract  award  (C3).86  The  expected  value  of  winning  the  competition  is  given  by  the 
following  equation: 

E(W)  =  Pwj  [PQ  -  C(Q,MOE)  -  Cb  -  Cs],  (3) 

The  second  missing  component  in  the  representative  bidder’s  profit  function 
is  the  expected  value  of  losing  the  competition,  say  E(L).  This  is  simply  the 
probability  of  losing  the  competition  (one  minus  the  probability  of  winning),  multiplied 
by  the  next-best  alternative  project,  Xo,  minus  the  bid  and  proposal  costs,  Cb,  such 
that 


E(L)  =  (1  -  Pw)[Xo  -  Cb]. 


(4) 


86  As  discussed  earlier,  the  expected  costs  of  defending  the  contract  award  can  be  critical  to  the 
decision  of  whether  or  not  a  winning  bidder  type  decides  to  participate.  The  easier  it  is  to  file  a  protest 
(i.e.,  the  lower  the  filing  costs)  and  the  greater  the  probability  of  winning  the  protest  (Pm  x  Ps),  the 
more  that  losing  bidder  types  will  be  attracted,  but  the  less  that  winning  bidder  types  will  show  up  (or 
else  charge  a  stiff  risk  premium). 
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Combining  Equations  (1)-(4)  yields  a  representative  bidder’s  profit  function, 

V.  The  bidder’s  problem  is  to  choose  a  price  to  charge  (P),  the  quality/schedule  to 
offer  ( MOE ),  and  a  prospective  protest  investment  (/)  to 

Maximize  V  =  Pw  [PQ  -  C(Q,MOE)  -  Cb  -  C3]  (winning  the  competition) 

+  (1  -  Pw)  [Xo  -  Cb]  (losing  the  competition) 

+  ( PmPs )  [Xa  -  Cb  -  Cf  -  /]  (winning  the  protest) 

+  (1  -Pm)[Xo-Cb-Cf-I-Ci]  +  Pm(  1  -  Ps)[Xo-Cb-Cf- I-C2] 

(losing  the  protest) 

Given  this  profit  function,  the  first  order  conditions  of  an  unconstrained 
optimization  allow  us  to  generate  some  intuitive  comparative  statics  results.  Applying 
the  Envelope  Theorem  reveals  the  impact  of  changes  in  the  variables  on  a 
representative  bidder’s  expected  profits  for  several  of  the  variables.  A  summary  of 
these  results  is  provided  in  Table  5,  and  a  discussion  appears  below. 


Table  5.  Comparative  Static  Results  of  the  Variables  on  Expected  Profits 


Variables 

the 

Department 

might 

Influence: 

E 

errors 

F 

fraud 

N 

bidders 

XA 

protest 

award 

CB 
bid  & 
proposal 
costs 

C1;C2 

penalties 

for 

frivolous  or 
loser  pays 

CF 

filing  fees 
&  other 
protest 
costs 

Q 

quantity 

required 

Xo 

Value  of 
Alternative 
Projects 

Impact  of 
an  Increase 

on 

Expected 

Profits 

9 

■ 

9 

■ 

- 

+ 

- 

- 

- 

+ 

+ 

The  greater  the  number  of  bidders,  bid  and  proposal  costs,  penalties,  or  filing 
fees,  the  tower  the  expected  value  of  profits  to  a  representative  bidder  from  the 
competition  ( V ).  The  greater  the  protest  award,  quantity  of  the  contract,  or  value  of 
alternative  projects,  the  greater  the  expected  value  of  profits  to  a  representative 
bidder  from  the  competition  ( V ). 

An  interpretation  of  the  representative  bidder  model  points  to  several  of  the 
potential  change  candidates  discussed  earlier.  Bidders  control  the  price  they  charge 
(P),  the  quality/schedule  they  offer  {MOE),  and  any  investment  they  might  make  to 
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win  a  protest  (/),  so  the  government  controls  the  remaining  variables  that  appear  in 
Figure  10. 

1.  Reducing  Expected  Benefits 

To  reduce  expected  benefits,  the  first  step  is  to  identify  feasible  ways  to  lower 
the  probability  of  successful  protests.  This  involves  reducing  agency  errors  (£),  and 
fraud  and  favoritism  (F).  This  can  be  accomplished  by  (i)  investing  in  training  and 
experience,  (ii)  linking  pay  and  promotion  to  successful  procurement  outcomes,  (iii) 
ensuring  the  transparency  of  the  assessment  criteria,  (iv)  ensuring  the  transparency 
and  accountability  of  the  evaluation  and  selection  process,  (v)  making  companies 
aware  of  the  low  probability  of  awards  being  overturned,  and  (vi)  substituting 
protests  for  alternatives  such  as  random  (internal  and  external)  audits  and 
investigations,  encouraging  alternative  dispute  resolution  (ADR),  etc.  Alternatively, 
the  probability  of  success  would  drop  if  the  GAO  narrowed  standing  (i.e.,  eligibility) 
or  raised  the  threshold  required  for  a  protest  to  have  merit. 

Whistle-blower  lawsuits  and  integrity  pacts  are  examples  of  decentralized 
third-party  governance  mechanisms.  Internal  audits,  external  audits,  alternative 
dispute  resolution  (ADR),  and  IG  investigations  are  examples  of  centralized 
oversight  mechanisms.  Centralized  and  decentralized  oversight  mechanisms  can  be 
either  complements  or  substitutes  for  protests,  but  regardless,  by  deterring  fraud  and 
errors,  they  will  lower  the  probability  of  a  successful  protest. 

Similarly,  carefully  crafted  human  capital  investments  (such  as  improving 
public-sector  pay  and  training  for  procurement  officials)  can  also  substitute  for  more 
costly  centralized  and  decentralized  oversight.  Valuable  human  capital  investments 
include  initiatives  to  build  integrity  (e.g.,  ethics  training,  clearly  stated  and  widely 
distributed  standards  of  conduct,  etc.),  acquisition  training  (e.g.,  augmenting  skills 
and  experience),  and  pay  for  performance  (or  other  ways  to  align  incentives  to 
improve  procurement  outcomes).  Internal  audits,  external  audits,  investigations, 
sanctions,  and  penalties  offer  other  approaches — all  of  which  represent  attempts  to 
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increase  transparency  and  accountability.  In  combination,  these  investments  offer  a 
potential  return  in  terms  of  reduced  fraud  and  errors  that  is  reflected  in  a  lower 
probability  of  a  successful  protest.  Investing  in  training,  linking  pay  and  promotion  to 
successful  procurement  outcomes,  and  hiring  more  experienced  acquisition 
personnel  will  reduce  protests  and  simultaneously  build  confidence  in  the  integrity  of 
the  solicitation  and  selection  process,  boosting  the  expected  benefits  of  participation 
and  therefore  the  benefits  of  competition. 

The  second  step  is  to  identify  feasible  ways  to  reduce  the  payoff  from  a 
successful  protest.  For  example,  by  (i)  reducing  the  award  (such  as  by  unbundling 
the  contract  vertically,  in  terms  of  different  stages  of  production,  or  horizontally,  in 
terms  of  quantities);  (ii)  sharing  the  award  (split  buys);  or  (iii)  keeping  records  of 
protests  by  firms  and  using  this  information  in  future  competitions  (i.e.,  using  a 
company’s  reputation),  etc. 

2.  Increasing  Expected  Costs 

To  increase  the  expected  costs  of  a  protest  requires  increasing  either  the 
probability  of  losing  a  protest  or  the  negative  payoff  associated  with  losing  the 
protest  (or  both).  Strategies  might  include  (i)  restricting  the  filing  window  or  raising 
filing  fees  to  reflect  the  government’s  (agency  and  GAO)  complete  (visible  and 
hidden)  costs  to  administer  a  protest  (Cf),  (ii)  requiring  firms  to  post  a  bond  (as  in  the 
case  of  the  COFC)  to  compensate  for  delays  in  the  event  the  protest  is  denied,  or 
(iii)  assessing  penalties  for  clearly  strategic  (frivolous/nuisance)  protests  and/or  for 
those  that  have  some  merit,  but  are  not  sustained,  e.g.,  loser  pays,  corresponding  to 
penalties  ( Ci ,  C2).  In  Figure  5. 

In  lowering  the  expected  benefits  from  a  protest  or  in  raising  the  expected 
costs,  the  challenge  is  to  choose  the  most  cost-effective  portfolio  of  initiatives  that 
aligns  incentives  and  that  adequately  monitors  and  enforces  procurements  in  order 
to  reduce  the  risk  of  fraud  and  errors  and  to  encourage  competition.  In  searching  for 
alternative  approaches,  the  optimum  mix  will  include  targeted  investments  in  human 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-71  - 


capital  as  well  as  centralized  (audits  and  investigations)  and  decentralized  (bid 
protests)  third-party  governance  mechanisms. 

F.  An  Application  of  the  Model:  The  KC-X  Procurement 

An  excellent  illustration  of  the  powerful  role  of  incentives  in  the  protest 
process  is  the  USAF’s  search  for  a  new  aerial  tanker  to  replace  the  KC-135  fleet. 

The  two  qualified  bidders  (Boeing  and  Northrop  Grumman-EADS  [NG-EADS])  both 
offered  mature  designs.  Yet  procurement  efforts  have  been  consistently 
disappointing.  In  this  case,  the  usual  explanation  of  insufficient  technical  maturity  is 
not  an  excuse.  Both  candidates  (the  KC-767  and  KC-30)  have  been  developed  for 
other  air  forces,  have  been  successfully  delivered  (deployed),  and  are  currently 
operational.  The  breakdown  has  been  in  the  conduct  of  the  U.S.  competition. 

Boeing’s  2008  protest  of  the  KC-45  contract  award  to  Northrop  Grumman- 
Eads  (the  KC-30  proposal)  was  a  serious  obstacle  in  finding  a  supplier  for  the  next 
USAF  aerial  tanker.  As  illustrated  in  the  representative  bidder  model  in  Figure  5 
above,  the  decision  to  protest  is  influenced  by  a  number  of  factors — especially  (a) 
the  probability  of  a  successful  protest  (i.e.,  the  probability  of  merit,  and  the 
probability  the  protest  is  sustained  given  that  it  has  merit),  (b)  the  expected  return  (or 
payoff)  from  a  successful  protest,  (c)  the  cost  of  conducting  a  protest  (i.e.,  filing  fees 
and  other  expenses,  including  opportunity  costs  of  the  assets  committed),  and  (d) 
the  value  of  the  next-best  alternative  project  available  to  the  firm. 

When  there  are  only  two  potential  KC-X  suppliers,  this  encourages  the 
strategic  use  of  protests  by  firms  as  a  bargaining  tool.  For  example,  either  firm  can 
hold  up  the  procurement  process  to  gain  strategic  advantage  by  issuing  a  threat  not 
to  respond  to  the  Request  for  Proposal  (RFP) — i.e.,  threatening  to  leave  the  DoD 
with  a  non-competitive  procurement  that  violates  the  spirit  of  the  CICA.  In  fact,  both 
Boeing  and  NG-EADS  have  successfully  used  similar  hold-up  strategies.  In  one 
instance  in  2007,  the  RFP  was  amended.  In  another  instance  in  2008,  the  post- 
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protest  attempt  to  re-compete  the  KC-X  contract  award  was  aborted.  The  latest  twist 
in  2010  is  that  one  of  the  candidates  threatened  not  to  respond  to  the  RFP. 

It  might  appear  that  it  should  be  possible  to  write  an  RFP  acceptable  to  both 
suppliers  that  would  result  in  a  competition  less  vulnerable  to  protest.  Unfortunately, 
the  RFP  “fairness”  constraint  likely  interferes  with  the  aim  of  conducting  a  protest- 
resistant  source  selection  process.87 

Although  it  may  be  useful  to  aspire,  as  the  Air  Force  Chief  of  Staff  has  said,  to 
a  “bulletproof”  RFP  (Gnau,  2009)  and  a  “perfecto”  competition  (Bennett,  2009) — with 
a  view  to  lowering  the  probability  of  a  successful  protest — this  is  only  part  of  the 
story.  The  other  part  is  to  understand  the  overwhelming  incentives  created  by  the 
extraordinary  benefits  (payoff)  of  the  KC-X  contract.  The  magnitude  of  the  contract 
strongly  encourages  either  company  to  protest  not  being  selected  even  if  there’s 
only  a  slight  chance  of  success.  At  the  DoD’s  insistence,  and  with  presidential 
support,  the  KC-X  competition  was  specifically  designed  to  award  a  large,  long-term, 
winner-take-all  contract.  The  unintended  consequence  has  been  a  strong  incentive 
for  the  losing  firm  to  protest  the  award — even  if  that  protest  is  unlikely  to  succeed. 

The  losing  firm’s  upside  is  the  expected  return  from  overturning  the  award,  in 
this  case,  a  historically  unprecedented  award  to  the  winning  firm — a  revenue  stream 
estimated  at  up  to  $1 00  billion  over  the  life  of  the  tanker  contract  (e.g.,  see 
Johnsson,  2009).  A  successful  protest  is  likely  to  result  in  an  amended  competition 
(or  in  an  entirely  new  competition).  Given  only  two  serious  competitors,  the  chances 
of  winning  a  new  competition  are  potentially  much  greater  than  if  multiple  firms  were 
in  competition.  In  this  case,  the  chance  (small  probability)  of  an  enormous  payoff 
virtually  guarantees  protests  unless  artificial  limits  (quotas)  are  imposed  on  the 
ability  to  protest  (similar  to  the  limited  number  of  reviews  allowed  on  line  calls  by 
players  in  professional  tennis). 


87  This  merits  further  study. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-73- 


Reinforcing  the  incentives  for  the  two  companies  to  protest  are  their 
calculations  of  the  expected  costs  of  conducting  a  protest  relative  to  the  expected 
benefits.  The  costs  include  (i)  legal  and  other  related  pecuniary  costs  (filing  fees, 
etc.),  and  (ii)  opportunity  costs  of  distracting  the  company  from  other  possible 
pursuits  and  of  preserving  the  management  team  and  technical  experts  organized 
for  the  original  competition. 

In  the  case  of  the  KC-X,  protest  costs  are  trivial  relative  to  the  size  of  the 
contract  (and  potential  profits).  Moreover,  the  opportunity  costs  of  keeping  the 
management  and  technical  teams  together  are  likewise  relatively  small.  These 
opportunity  costs  include  the  loss  of  opportunities  to  use  that  team  to  compete  for 
other  business.  In  this  case,  the  losing  bidder  is  very  likely  to  be  forced  out  of  the 
new  tanker  market  for  well  over  a  decade,  suggesting  that  regardless  of  other 
opportunities,  the  team’s  opportunity  costs  of  pursuing  the  contract  will  be  relatively 
small. 


In  short,  even  if  the  USAF  was  able  to  take  significant  steps  to  reduce  the 
probability  of  a  successful  protest,  or  to  substantially  raise  the  costs  of  a  protest,  the 
potential  payoff  is  so  large  that  either  firm  would  still  face  positive  net  expected 
benefits  from  protesting  if  they  lost  the  competition.  But  there’s  more  to  the  situation. 
If  the  award  is  re-competed  and  that  competition  turns  out  badly,  multiple  protests 
are  still  possible;  that  is,  it’s  possible  to  once  again  overturn  the  award  (as  in  the 
case  of  the  CSAR-X).  Moreover,  if  the  GAO  does  not  return  a  favorable  verdict,  then 
it  is  possible  for  the  protester  to  have  a  second  chance  at  protesting  the  award  at  the 
Court  of  Federal  Claims  (COFC)  and  to  be  successful  there  (similar  to  the  case  of 
the  KC-135  support  contract).88 


88 

This  suggests  investigating  alternate  strategies  of  dealing  with  the  dispute  before  it  takes  off.  For 
instance,  Boeing’s  supporters  in  Congress  used  the  GAO  sustainment  of  Boeing’s  protests  in  2008  as 
the  occasion  to  conduct  hearings  on  pre-award  issues  such  as  whether  USAF  solicitation  criteria 
properly  took  into  account  defense  industrial  base  impacts  and  foreign  subsidies  for  the  Northrop 
Grumman-EADS  team.  The  Air  Force  subsequently  cancelled  the  protested  solicitation  altogether. 
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In  this  case,  it  appears  the  only  real  solutions  to  the  problem  are:  to  reduce 
the  expected  benefits  (i.e.,  the  payoff)  from  protests  (including  from  the  automatic 
stay  of  the  procurement),  to  limit  the  grounds  or  number  of  protests  allowed  (which 
could  be  ruled  unconstitutional),  to  require  offerors  to  engage  in  ADR,  and/or  to 
manage  the  protest  process  in  ways  that  ensure  protesters  do  not  obtain  pre-award 
re-competition  remedies  with  post-award  arguments  (e.g.,  by  refusing  to  follow  the 
GAO  recommendation). 

Intense  competition  by  a  limited  set  of  suppliers  over  relatively  few  major 
defense  contracts  provides  a  good  business  case  for  a  losing  bidder  to  protest  even 
when  the  probability  of  success  is  quite  small.  Since  the  value  of  the  contract  is  one 
of  the  few  variables  the  Department  controls,  this  suggests  investigating  alternative 
approaches  to  reduce  the  size  of  the  contract,  or  else  accepting  the  likelihood  of 
persistent  protests  which  further  delay  procurements.  Strategies  might  include 
unbundling  the  contract  to  reduce  the  payoff  or  sharing  the  contract  (a  split  buy). 
ADR  is  particularly  conducive  to  reaching  a  split  buy  outcome. 

The  split-buy  option  can  be  combined  with  a  clever  incentive  mechanism  to 
preserve  competition  over  time.  This  study  recommends  that  the  split  depend  on  the 
relative  value  of  the  contract  offers,  with  the  winner  receiving  a  larger  split  (i.e., 
share  of  the  procurement),  the  greater  the  difference  between  the  highest  and 
second-highest  valued  contractor.  For  example,  if  the  second-best  offer  is  close 
enough  to  the  first-best  offer,  the  contract  split  would  be  close  to  50:50.  If  structured 
correctly,  such  a  split  strategy  has  the  potential  to  retain  the  benefits  of  competition. 

It  encourages  contractors  to  submit  proposals  that  reflect  what  they  view  as  the  best 
value  they  can  provide  to  the  government  so  that  they  obtain  the  largest  possible 
share  of  the  contract.  Contract  shares  could  be  adjusted  over  time  to  reward 
contract  performance  as  contractors  invest  in  cost-reducing  technologies  or 
experience  different  learning  curves.  Split  buys  also  increase  competition  for  follow- 
on  contracts  by  preserving  multiple  procurement  sources.  This  offers  a  constructive 
approach  to  discourage  costly  bid  protests  and  deserves  more  careful  consideration. 
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Many  of  the  challenges  associated  with  bid  protests  in  the  KC-X  procurement 
are,  in  a  very  real  sense,  a  natural  outcome  of  DoD  policies.  Secretary  of  Defense 
William  Perry’s  “last  supper”  meeting  with  the  CEOs  of  major  defense  companies 
after  the  end  of  the  Cold  War  (and  demise  of  the  Soviet  Union)  facilitated  a 
significant  consolidation  of  the  American  defense  industrial  base.  An  unintended 
consequence  of  reducing  the  number  of  bidders  may  have  been  to  make  the 
procurement  of  major  complex  weapons  systems  more  prone  to  protests.  Large, 
winner-take-all  competitions  have  also  created  strong  incentives  to  protest  contract 
awards. 

The  KC-X  tanker  competition  illustrates  another  problem  related  to  bid 
protests:  missed  opportunities  to  manage  the  GAO  protest  process  in  ways  that  are 
allowed  under  current  protest  regulations.  Protesters  such  as  Boeing  can  exploit 
agency  errors  and  missed  opportunities  by  pursuing  post-award  protests  in  order  to 
obtain  a  pre-award  remedy — for  example,  a  new,  “start-from-scratch”  competition 
(which  imposes  delays  that  are  in  the  protestor’s  interest).  The  Air  Force  could  have 
reduced  the  likelihood  of  this  outcome  by  using  early  alternative  dispute  resolution 
forums,  issuing  stay  overrides,  or  by  refusing  to  follow  GAO  recommendations  in 
whole  or  in  part.89 

Finally,  bid  protests  are  best  viewed  as  part  of  a  larger  strategic  game,  which, 
includes  possible  efforts  by  competitors  to  re-shape  RFPs  (to  influence  the  executive 
branch),  to  engage  in  award  protests  (to  achieve  success  at  the  GAO,  part  of  the 
legislative  branch),  to  pursue  litigation  in  federal  courts  (to  achieve  success  through 
the  judicial  branch),  and  to  lobby  members  of  Congress.  In  the  latter  case,  stretching 
out  the  procurement  process  through  protests,  and  other  litigation,  affords  time  to 
mobilize  political  support — particularly  for  high-stakes  awards  (like  the  KC-X). 


89  Potential  strategies  and  tools  for  expeditious  and  less  costly  resolution  of  protests  are  addressed  in 
the  companion  report  by  Benishek  et  al  (201 1 ). 
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Indeed,  both  Boeing  and  NG-EADS  have  been  very  successful  in  mobilizing  special 
interests  to  build  strong  support  in  Congress  for  their  respective  proposals. 


,NPS 
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IV. 


Summary  and  Selected  Recommendations 


The  goal  of  this  study  was  to  offer  senior  decision-makers  a  useful  framework 
and  perspective  to  evaluate,  articulate,  and  recommend  modifications  of  the 
government’s  bid  protest  policy  to  improve  procurement  outcomes.  A  key 
contribution  is  to  add  an  economics  dimension  to  a  process  mostly  thought  of  in 
legal  terms. 

The  legal  and  regulatory  literature  that  underpins  the  current  protest  system 
reflects  two  fundamental  beliefs:  Protests  (a)  deter  fraud  and  errors  in  government 
procurements  and  (b)  promote  competition.  This  study  explored  the  first  premise 
through  the  economist’s  principal-agent  framework  and  the  second  with  the  aid  of  a 
representative  bidder  model.  The  aim  was  to  examine  opportunities  to  improve  the 
efficiency  and  effectiveness  of  the  protest  process  in  order  to  enhance  performance, 
cost,  and  schedule  outcomes  of  Defense  Department  acquisition  investments. 

In  economics,  there  is  no  such  thing  as  a  free  lunch.  Operating  a  protest 
system  is  costly.  A  fundamental  challenge  identified  by  the  Defense  Department  is 
the  significant  costs  of  protests  to  the  department,  which  is  reflected  in  the  recent 
experience  of  selecting  a  vendor  for  the  Air  Force’s  KC-X  aerial  tanker. 

Along  with  the  well-documented  benefits  of  allowing  firms  to  protest  public 
procurements  come  significant  opportunity  and  transaction  costs.  The  challenge  is  to 
reduce  the  risk  of  protests  of  critical  acquisitions  that  result  in  significant  cost 
overruns,  schedule  delays,  and  performance  gaps,  while  preserving  the  benefits 
offered  by  protests  to  promote  competition  and  ensure  the  integrity,  transparency, 
and  accountability  of  the  procurement  process. 

Adopting  an  economics  perspective  reveals  two  other  crucial  insights:  first, 
protests  can  be  exploited  by  firms  as  a  strategic  bargaining  tool  (Fed-mail);  and 
second,  governments  can  overreact  in  attempts  to  achieve  protest-proof 
procurements  (or,  alternatively,  exhibit  tacit  consent  to  protests — buy-offs).  These 
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hidden  costs  significantly  impact  acquisitions.  Combined,  they  substantially  increase 
risks  of  delays  and  cost-overruns. 

Fortunately,  fighting  agency  errors  and  corruption  and  promoting  competition 
can  be  done  in  other  ways.  The  economics  approach  encourages  senior  leadership 
to  identify  and  evaluate  alternatives  to  protests  that  could  achieve  similar  results  at  a 
lower  cost.  The  importance  of  acknowledging  the  substantial  costs  of  operating  a 
protest  system  to  govern  procurement  and  encourage  competition  is  one  of  the  key 
insights  of  the  study:  There  exists  an  optimum  rate  of  protests.  The  optimum  rate  of 
protests  offers  the  most  efficient  and  effective  way  to  encourage  competition  and 
ensure  the  integrity  of  the  acquisition  process  while  guaranteeing  the  best  possible 
outcomes  for  our  troops  and  taxpayers. 

Although  it  is  not  easy  to  locate  this  optimum,  the  economics  approach  can 
help  identify  and  evaluate  policies  that  move  us  closer  to  the  optimum.  One 
important  implication  is  that  a  simple,  often-stated  objective  such  as  “reducing  the 
rate  of  successful  protests”  is  not  always  appropriate.  Two  additional  factors  need  to 
be  considered.  The  first  is  to  identify  where  the  department  is  relative  to  the 
optimum.  The  second  involves  a  careful  consideration  of  how  the  department  might 
choose  to  reduce  protests.  In  recognizing  the  costs  as  well  as  benefits  of  a  protest 
system,  the  economics  approach  suggests  that  reviewing  portfolios  of  governance 
mechanisms  (internal  audits,  external  audits,  independent  investigations,  ADR, 
integrity  pacts,  training,  incentives,  etc.)  can  substitute  for  (or  complement)  protests 
to  improve  procurement  outcomes. 

If  the  Defense  Department  is  facing  protests  above  the  optimum  rate  ,  then 
the  burden  outweighs  the  benefits  and  reducing  the  rate  of  protests  is  appropriate. 
This  can  be  accomplished  in  two  ways:  by  reducing  the  expected  benefits  of  a 
protest  to  a  disappointed  bidder  or  by  increasing  the  expected  costs.  This  is  a  crucial 
distinction  emphasized  throughout  the  study. 
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If  the  Defense  Department  is  below  the  optimum  rate,  then  the  benefits  of 
protests  outweigh  the  extra  (opportunity  and  transaction)  costs.  In  this  range,  while 
reducing  protests  might  appear  to  have  positive  short-run  impacts,  it  could  be  at  the 
expense  of  long-term  benefits  such  as  preserving  the  integrity  of  the  procurement 
process  and  could  sacrifice  future  competition.  Remarkably,  this  report  revealed  two 
important  cases  that  demonstrated  that  even  in  this  range,  it  is  possible  to  reduce 
protests  and  improve  long-term  outcomes.  The  key  is  to  carefully  consider  how  the 
department  reduces  protests. 

This  study  captured  insights  from  the  protest  literature  and  offered  a  brief 
overview  of  the  current  protest  process;  it  also  discussed  alternative  governance 
mechanisms.  Woven  throughout  were  data,  examples,  and  survey  results.  Two 
economic  frameworks  were  introduced — the  principal-agent  model  (agency  theory) 
and  a  representative  bidder  model.  These  frameworks  revealed  two  sets  of  potential 
change  candidates — factors  that  reduce  the  expected  benefits  to  a  losing  bidder  of  a 
protest  and  factors  that  raise  the  expected  costs. 

In  conclusion,  many  countries  allow  disappointed  bidders  to  protest  public 
procurements  as  an  oversight  mechanism  to  minimize  fraud  and  errors  by 
procurement  officials.  The  belief  is  that  allowing  disappointed  bidders  to  protest 
increases  integrity,  transparency,  and  accountability,  and  encourages  competition, 
reducing  the  cost  of  public  purchases.  A  key  contribution  of  this  study  is  to  add  an 
economics  dimension  to  a  protest  process  mostly  thought  of  in  legal  terms.  The 
economics  approach  suggests  companies  are  likely  to  use  the  protest  process 
strategically  to  improve  their  competitive  bargaining  position  and  will  be  equally 
strategic  in  their  decision  to  protest  an  award.  The  goal  for  governments  is  to 
minimize  the  risk  of  protests  that  result  in  cost  overruns,  schedule  delays,  and 
performance  gaps,  while  preserving  benefits  offered  by  protests  to  promote 
competition  and  ensure  the  integrity,  transparency  and  accountability  of  the 
procurement  process.  Adopting  an  economics  perspective  reveals  two  other  crucial 
insights:  firms  can  exploit  protests  to  extract  concessions  (Fed-mail),  and  risk-averse 
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governments  can  overreact  in  their  attempts  to  achieve  protest-proof  procurements 
(buy-offs).  In  recognizing  the  costs  as  well  as  the  benefits  of  a  protest  system,  the 
economics  approach  invites  a  review  of  alternative  portfolios  of  governance 
mechanisms  that  complement  bid  protests  to  improve  procurement  outcomes. 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-82- 


V. 


Appendix  1:  Selected  Observations  and 
Recommendations 


According  to  Francis,  Golden,  and  Woods  (2010), 

In  Fiscal  year  2009,  the  DoD  spent  nearly  $384  billion  on  contracts  for  goods 
and  services,  ...  representing  over  70  percent  of  total  government  contract 
spending.  [This]  makes  it  imperative  for  DoD  to  manage  risk  in  all  of  its 
acquisitions  to  ensure  that  the  government’s  best  interests  are  being  met.  ... 
Clearly  however,  DoD  has  not  adequately  managed  such  risks.  For  example, 
its  major  weapons  systems  continue  to  take  longer  to  develop,  cost  more,  and 
deliver  fewer  quantities  and  capabilities  than  originally  planned,  (p.  1 ) 

A  question  is  whether  the  GAO  or  the  agencies  themselves  might  be 
inadvertently  contributing  to  some  of  these  challenges  through  the  bid  protest 
oversight  process. 

Poorly  defined  or  changing  requirements  have  contributed  to  increased  costs, 
as  well  as  services  that  did  not  meet  the  department’s  needs  (p.  4).  ... 

Recently,  DoD  established  a  multiphased,  peer-review  process  for  services 
acquisitions  intended  to  ensure  consistent  and  appropriate  implementation  of 
policy  and  regulations,  improve  the  quality  of  contracting  processes,  and 
facilitate  sharing  best  practices  and  lessons  learned  (p.  5).  ... 

One  of  the  principal  tools  for  ensuring  the  integrity  of  the  competition  system 
is  the  bid  protest  process.  ...  [W]hen  viewed  historically,  ...  the  number  of 
protests  challenging  contract  awards  in  the  last  5  years  is  relatively  low.  ... 
The  bid  protest  process  involves  a  legal,  adjudicative  function;  both  the 
process  and  the  resulting  product  differ  from  those  associated  with  the  reports 
that  GAO  issues  in  connection  with  its  program  audits  and  reviews  (p.  9).  ... 

[A]  protest  before  GAO  ...  does  not  address  broad  programmatic  issues,  such 
as  whether  or  not  a  weapons  program  is  being  managed  effectively  or 
consistent  with  best  practices.  Our  decision  ...  reaches  no  conclusion  about 
which  of  the  offered  goods  or  services  will  best  meet  the  agency’s  needs. 
[EXCEPT  FOR  DETERMINING  PREJUDICE]  Instead,  a  bid  protest  decision 
addresses  specific  allegations  raised  by  an  unsuccessful  offeror  challenging 
particular  procurement  actions  ...  contrary  to  procurement  laws,  regulations, 
or  the  evaluation  scheme  set  forth  in  the  solicitation.”  We  sustain  a  protest 
when  we  find  that  the  contracting  agency  has  not  complied  with  procurement 
laws,  regulations,  or  the  solicitation’s  evaluation  scheme,  and  that  this  error 
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prejudiced  the  protester’s  chances  of  winning  the  competition  for  the  contract. 
Evaluating  offerors  in  a  manner  consistent  with  the  solicitation’s  stated  ground 
rules  is  a  requirement  for  conducting  an  impartial  and  objective  procurement 

(p.  10).  ... 

[Tjhere  are  other  reasons  why  GAO  might  sustain  a  protest.  For  example,  an 
agency’s  evaluation  may  not  be  adequately  documented,  or  the  evaluation 
conclusions  may  not  be  supported  by  the  record.  An  agency  also  may  have 
conducted  inadequate  or  misleading  discussions,  or  evaluated  offeror’s 
proposals  in  a  disparate  manner  (p.  11).  ... 

GAO’s  bid  protest  process  reduces  potential  disruptions  to  DoD  procurements 
as  a  result  of  three  factors:  (1 )  GAO  consistently  closes  more  than  50  percent 
of  all  protests  ...  within  30  days  of  filing;  (2)  remaining  DoD  protests  must  be, 
and  are,  resolved  with  100  days;  ...  (3)  CICA  permits  agencies  to  proceed 
with  contract  performance  even  before  a  protest  is  resolved  when  the  goods 
or  services  are  urgently  needed,  or  when  proceeding  is  in  the  best  interests  of 
the  U.S.  (p.  11).  ... 

In  short,  while  there  are  challenges  associated  with  balancing 
competing  interests  inherent  in  the  protest  system,  public  bid  protest 
decisions  ...  [provide]  transparency  in  the  procurement  system  and 
guidance  to  the  procurement  community,  without  undue  disruption  to 
the  acquisition  process  (p.  12).  ... 

The  President’s  March  4,  2009,  memorandum  ...  addresses  the  need  for 
improvement  in  the  procurement  system.  ...  [T]he  administration  has  tasked 
agencies  with  making  measurable  improvements  in  four  key  areas:  (1) 
increasing  competition,  (2)  reducing  the  use  of  high-risk  contracting 
approaches,  (3)  improving  the  acquisition  workforce,  (4)  determining  the 
appropriate  use  of  contractors  versus  federal  employees  (p.  12).  ... 

There  has  been  a  substantial  increase  in  spending  on  acquisition  programs 
and  services,  while  the  number  of  civilian  and  military  personnel  in  DoD’s 
acquisition  workforce  has  remained  relatively  constant,  ...  workforce  shortfalls 
have  resulted  in  degradation  of  oversight  ...  and  increased  workloads  for 
existing  staff.  To  supplement  its  ...  acquisition  workforce,  DoD  relies  heavily 
on  contractor  personnel.  ...  Without  an  adequate  workforce  ...  there  is  an 
increased  risk  of  poor  acquisition  outcomes  and  vulnerability  to  fraud,  waste, 
and  abuse  (p.  15).  ... 

[DoD]  is  conducting  a  competency  assessment  to  identify  the  skill  sets  of  its 
current  in-house  acquisition  workforce  and  increasing  the  size  of  its 
acquisition  workforce  (p.  16). 
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The  FY  201 1  budget  includes  $864  million  in  RDT&E  for  developing  a  new 
generation  tanker,  which  remains  a  very  high  priority.  Initial  plans  call  for 
procuring  179  commercial-derivative  tanker  aircraft  to  replace  roughly  a  third 
of  the  current  aerial  refueling  tanker  fleet  at  a  projected  cost  of  $35  billion 
(DoD,  2010,  p.  5)  ... 

To  make  the  most  of  the  nation’s  resources  entrusted  to  the  DoD  and  to  carry 
out  its  mission  more  effectively,  the  Department  continues  striving  to  improve 
the  way  it  does  business  (p.  6).  ... 

A  critical  effort  involves  reforming  the  processes  by  which  we  buy  weapons 
and  other  important  systems  through  implementation  of  the  Weapon  System 
Acquisition  Reform  Act.  ...  [The]  Department  is  increasing  the  number  of 
acquisition  personnel  by  20,000.  ...  The  FY  201 1  budget  request  continues 
support  for  this  transition  and  includes  funds  for  training  and  retention 
programs  that  will  bolster  the  capability  and  size  of  the  acquisition  workforce. 
(P-6)... 

A.  Common  Bases  for  Protests 

In  a  January  2009  report,  the  USAF  published  the  most  common  grounds  for 
filing  a  protest  that  it  sees:  “The  most  common  bases  for  protest  are:  improper 
evaluation  of  proposals,  defective  solicitations,  improper  cancellations  of 
solicitations,  failure  to  properly  compete  acquisitions,  and  failure  to  properly 
implement  socio-economic  goals  (e.g.,  small  business  set  asides)”  (USAF,  2009 
p.  8).  These  causes  for  protest  are  often  based  on  rules  the  USAF  introduces  in  its 
Requests  for  Proposals  (RFPs)  and  that,  in  some  cases,  it  does  not  follow. 

Several  complaints  (bid  protests)  might  appear  under  a  general  heading  that 
might  be  called  errors  ( E  in  the  model):  According  to  recent  studies  (JAT,  2009; 
Gansler,  Lucyshyn,  &  Arendt,  2009),  the  most  common  errors  involve  the  following: 

i)  Defective  Solicitations:  These  often  take  the  form  of  poorly 
constructed  RFPs,  usually  through  omitted  information  or 
contradictions  within  the  solicitation.  It  does  not  necessarily 
deal  with  how  the  agency  evaluated  competitors  for  the 
contract,  but  rather  how  the  agency  solicited  bidders  and  how  it 
constructed  the  requirements  for  the  desired  goods  or  services 
(E). 
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ii)  Improper  Evaluations:  The  improper  evaluation  of  proposals 
can  take  several  different  forms,  including  unequal  cost 
evaluations,  relative  weights,  or  technical  evaluation  factors. 

The  common  factor  in  improper  evaluation  claims  is  that  the 
protestor  felt  the  government  agency  violated  at  least  one 
procurement  regulation  and,  as  a  result,  prejudiced  the 
protesting  company  (E). 

iii)  Improper  Cancellations:  Claims  of  improper  cancellation  of  a 
solicitation  often  occur  after  an  agency  solicits  bidders  for  a 
contract  and  then  cancels  the  RFP  because  of  a  shift  in  demand 
(perhaps  due  to  changes  in  the  battlefield),  or  in  an  effort  to 
conserve  valuable  government  resources  for  higher  priority 
projects  (E). 

Defective  solicitations  and  improper  cancellations  boil  down  to  the 
government  not  offering  the  solicitation  to  the  right  competitors.  Either  it  did  not 
consider  a  legitimate  competitor  based  on  reputation  or  on  some  other  factor,  or  it 
may  have  violated  small-business  regulations  by  offering  the  solicitation  to  ineligible 
competitors  based  on  their  size  (E). 

Another  common  reason  (error)  for  filing  a  protest — aside  from  improprieties 
in  the  evaluation,  in  the  RFP,  or  in  source  selection — is  the  failure  to  conduct  fair  and 
meaningful  discussions  with  competitors  (SSJAT,  2009,  pp.  9-10).  During  the 
contracting  process,  the  agency  representative  must  treat  all  competitors  fairly  and 
impartially.  If  an  agency  is  found  to  have  conducted  unfair  discussions  with  a 
competitor  (such  as  only  contacting  one  competitor  or  revealing  evaluation 
information  to  one  competitor  but  not  to  all  that  could  give  an  unfair  advantage 
[fraud]),  the  award  of  contract  will  likely  be  protested  (E  for  error  and  F  for  fraud  in 
the  model). 

b.  Recent  Increase  in  the  Number  of  Protests  Filed 

Based  on  the  information  in  a  2009  GAO  report  to  Congress  (p.  7),  the 
number  of  DoD  protests  filed  increased  from  458  in  FY2004  to  61 1  in  FY2008  (see 
Figure  6).  This  increase  in  the  number  of  protests  filed  against  DoD  agencies  may 
or  may  not  be  statistically  significant,  but  it  suggests  that  the  number  of  protests  filed 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-86- 


each  year  against  the  DoD  recently  increased.90  Several  potential  factors  could 
have  contributed  to  this  recent  increase  in  protest  filings. 


Figure  6.  Number  of  DoD  Protests  Filed  FY2004-FY2008 

(GAO,  2009) 

First,  if  the  government  does  not  follow  its  own  rules  when  evaluating 
competitors’  proposals,  it  will  likely  see  more  businesses  protesting  contracting 
decisions.  This  is  the  simplified  reason  for  why  the  KC-X  protest  was  sustained 
(SAF  AQ,  15  September  2008,  pp.  7-9) — the  Air  Force  did  not  follow  the  rules  it  set 
forth  in  the  proposal  solicitation  with  regard  to  performance  evaluation  weights,  cost 
evaluation  methods,  and  several  other  self-imposed  guidelines.  If  an  agency  has 
trouble  following  the  guidelines  that  it  sets  forth  in  its  RFP,  then  companies  will  be 
more  likely  to  protest  its  evaluations  and  the  award  of  the  contract  (E). 

A  second  reason  for  the  apparent  increase  in  protests  could  be  the  high  ops 
tempo  of  today’s  Air  Force  combined  with  an  overloaded  contracting  workforce.  The 
United  States  has  been  fighting  two  undeclared  wars  for  almost  a  decade,  which  has 


90  Even  more  interesting  for  future  research  is  to  discover  why  there  was  such  a  significant  drop  in 
protests  filed  at  the  GAO  over  the  period  1994-2001,  as  illustrated  in  Figure  13. 
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caused  an  increase  in  workload  for  contracting  officials.  As  of  FY2008,  the  average 
contracting  official  was  responsible  for  roughly  $20  million  of  taxpayer  money 
(SSJAT,  2009,  Slide  22),  roughly  three  times  the  level  of  responsibility  per  worker  in 
FY2001 .  Since  the  average  worker  is  now  responsible  for  more  taxpayer  money 
than  in  the  past,  it  is  possible  that  details  are  being  missed  on  these  larger,  more 
complex  contracts,  which  can  lead  to  more  protests  (E). 

Another  reason  for  the  recent  increase  in  protests  is  that  the  government  is 
moving  toward  larger,  less  frequent  procurements.  One  example  is  Alliant,  “a  10- 
year,  $50  billion  contract  that  replaces  three  expiring  contracts” 
(http://www.federaltimes. com/index. php?S=402260)  {XA  and  Xo  in  the  model). 

Because  the  stakes  of  winning  a  contract  are  high,  it  is  critical  for  companies 
to  win  those  contracts  to  survive  financially;  it  also  should  be  expected  that  in  order 
to  have  another  shot  at  the  prize,  companies  who  lose  a  contract  will  protest  if  they 
felt  prejudiced  (X4;  Xo). 

Also,  some  contracts  are  only  renewed  once  every  several  years — for 
example,  the  Air  Force  will  not  need  a  new  tanker  each  year,  so  the  winner  of  that 
contract  will  have  a  guaranteed  customer  base  for  a  significant  period.  Companies 
who  win  government  contracts  will  have  an  advantage  over  their  competitors  for  the 
life  of  those  contracts,  so  government  contracts  are  prizes  worth  fighting  for  via 
protest  in  the  event  that  an  opponent  is  chosen  for  the  award  ( XA ;  Xo). 

As  indicated  several  times  in  this  report,  companies  will  likely  file  protests  if 
the  expected  benefits  of  filing  outweigh  the  expected  costs.  Assuming  the  end  goal 
for  any  company  is  to  maximize  the  value  to  shareholders,  the  decision  to  protest  in 
the  hopes  of  being  reconsidered  for  the  award  of  a  multimillion  dollar  contract  must 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 


-88- 


outweigh  the  cost  of  filing  in  addition  to  the  opportunity  cost  of  using  those  resources 
elsewhere  to  increase  the  odds  of  winning  other  contracts.91 

Another  possible  contributing  factor  to  the  recent  increase  in  protests  filed  is 
the  increase  in  DoD  procurement  spending.  As  previously  mentioned,  contractors 
are  responsible  for  almost  three  times  as  many  taxpayer  dollars  today  as  they  were 
eight  years  ago.  This  increased  workload  can  be  attributed  to  increases  in  defense 
contract  spending  over  the  last  decade.  From  FY2000-FY2008,  DoD  contract 
spending  increased  from  $140  billion  to  $325  billion,  as  shown  in  Figure  7  ( 
Schwartz,  2009,  p.  16).  As  the  DoD  spends  more  money  on  contracting  by 
increasing  both  the  number  of  contracts  and  the  dollar  value  of  those  contracts,  from 
the  representative  bidder  model,  it  follows  that  as  the  award  ( XA )  increases,  the 
number  of  contract  protests  will  also  increase.92 


91  Other  costs  exist  for  companies,  including  possible  loss  of  reputation  and  future  consideration  on 
contracts,  but  consider  for  a  moment  the  monetary  cost  of  filing  protests.  According  to  Watson  & 
Associates,  LLC,  a  business  law  practice,  the  cost  of  attorney’s  fees  for  filing  a  protest  typically 
average  about  $4,500.  For  large  companies,  this  cost  is  trivial  in  relation  to  the  potential  benefit  of 
winning  a  multimillion  (or  billion)  dollar  protest.  However,  for  smaller  businesses,  this  cost  might  be 
significant  and  could  strongly  impact  a  decision  of  whether  to  file  a  protest.  This  idea  could  be  tested 
by  examining  protest  data  and  the  net  worth  of  companies  that  file  protests  to  see  if  they  are 
positively  correlated. 

92  A  final  contributing  factor  to  the  recent  increase  in  the  number  of  protests  filed  is  the  increase  in 
GAO  jurisdiction  on  what  kinds  of  cases  it  can  hear.  “Congress  expanded  GAO’s  jurisdiction  in 
FY2007  to  include  hearing  protests  on  task  orders,  A-76  contracts,  and  Transportation  Security 
Administration  contracts”  (Schwartz,  2009,  p.  7).  This  statutory  expansion  of  the  GAO’s  bid  protest 
jurisdiction  could  be  linked  to  the  recent  increase  in  bid  protest  filings. 
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Source:  Schwartz  (2009). 

Figure  7.  DoD  Contract  Spending,  FY2000-FY2008 

It  is  important  to  note  that  despite  the  recent  increase  in  bid  protests,  the 
long-run  trend  is  favorable.  The  number  of  protests  challenging  DoD  contracts  over 
the  last  five  years  is  relatively  low  when  examined  over  the  last  twenty  years.  In 
FY1989,  1,490  protests  were  filed  against  the  DoD;  in  FY2008,  only  611  protests 
were  filed,  a  59%  decrease  over  this  twenty-year  period  (see  Figure  8;  GAO,  2009, 
p.  7). 


Thus,  while  the  number  of  protests  filed  in  recent  years  has  seen  a  relatively 
small  increase,  the  number  of  protests  filed  when  viewed  historically  is  less  than  half 
of  what  it  was  twenty  years  ago.  This  suggests  that  while  there  are  always 
improvements  that  can  be  made,  the  procurement  system  seems  to  be  working  the 
way  it  should  be  and  that  it  has  been  improving  over  the  last  two  decades  by 
reducing  the  number  of  protests  filed.93  There  is  no  conclusive  evidence  that  during 
the  last  eight  years  the  number  of  decisions  with  merit  have  significantly  increased. 


93  Between  FY2001  and  FY2008  the  number  of  merit  decisions  for  GAO  protests  stayed  relatively 
constant  around  300  decisions.  During  that  same  time  period,  DoD  merit  decisions  stayed  constant 
around  175  decisions  (see  Figure  9). 
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Figure  8.  Number  of  Protests  Filed  to  the  GAO,  FY1989-FY2008 

(GAO,  2009) 

The  remedy  offered  in  an  agency  protest  can  either  be  monetary  (such  as 
paying  attorney’s  fees  and/or  the  cost  of  filing  a  protest,  as  recommended  by  the 
GAO  in  the  event  of  a  sustain  decision)  to  compensate  the  protestor,  or  can  be  the 
voluntary  re-solicitation  for  proposals  or  reopening  of  discussions  in  order  to  correct 
the  prejudice  the  agency  created.  This  non-monetary  form  of  compensation  costs 
the  agency  time,  money,  and  other  resources  that  can  delay  projects  and  adversely 
impact  the  mission. 

Recall  the  earlier  recommendation  that  the  GAO  rethink  its  measure  called 
the  effectiveness  rate  since  it  can  easily  incorporate  Fed-mail  and  Buy-offs. 
Nevertheless,  several  potential  root  causes  have  been  identified  to  explain  increases 
in  the  effectiveness  rate.  According  to  an  April  2009  subcommittee  report  to  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  five  possible 
root  causes  explain  the  increase  in  effectiveness  rates  (JAT,  2009,  pp.  3-4). 

The  first  cause  identified  is  acquisition  complexity.  Since  FY2001 ,  the 
number  of  large  cost  complex  procurements  that  typically  involve  performance- 
based  contracts  have  steadily  increased  (JAT,  2009,  p.  3).  Larger  and  more 
complex  procurements  increase  the  likelihood  of  mistakes,  which  is  directly 
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proportional  to  the  number  of  protests  filed.  As  more  protests  are  filed,  more  aid  will 
be  given  to  protestors — either  through  sustain  decisions  or  voluntary  aid — which  will 
increase  the  effectiveness  rate. 

The  second  possible  root  cause  for  the  increase  in  the  effectiveness  rate  is 
an  increase  in  ops  tempo. 

The  number  of  contract  actions  being  processed  including  procurements 
related  to  urgent  operational  requirements  have  grown  significantly  in  the  past 
several  years  due  to  enhanced  military  operations  worldwide  and  associated 
deployments.  This  increase  in  workload  makes  the  procurement  process 
more  susceptible  to  mistakes.  (JAT,  2009,  p.  3) 

This,  in  turn,  leads  to  an  increase  in  effectiveness  rate.  The  DoD  needs  quick 
resolution  of  protests  in  order  to  have  the  promised  goods  or  services  of  the  contract 
delivered.  As  a  result,  it  is  possible  that  the  DoD  would  be  inclined  to  voluntarily  give 
aid  to  protestors  in  order  to  speed  the  process  along  to  reach  a  settlement  and 
complete  the  mission. 

Risk  aversion  is  another  possible  root  cause  for  the  increase  in  the 
effectiveness  rate.  “The  increase  in  voluntary  corrective  actions  (including  Fed-mail 
and  Buy-offs)  may  be  an  indication  that  DoD  activities  are  less  willing  to  let  GAO 
render  a  decision  that  could  bring  criticism  to  their  agency”  (JAT,  2009,  p.  3). 
Litigation  risk  is  a  large  concern  for  government  agencies  because  the  threat  of 
litigation  risks  large  amounts  of  valuable  resources  such  as  time,  money,  and 
manpower.  These  resources  could  be  put  to  use  elsewhere,  so  avoiding  litigation  is 
ideal.  Unfortunately,  literature  review  and  survey  data  in  the  companion  report 
suggest  that  agencies  are  failing  to  avoid  litigation  to  the  extent  allowed  by  current 
laws  and  regulations  (Benishek  et  al. ,  201 1 ,  pp.  45-58). 

Another  factor  identified  is  inadequate  manpower  due  to  a  large  workload. 

Although  the  total  number  of  contracting  personnel  remained  relatively 
constant  from  FY1998  to  FY2008,  the  workload  carried  by  contracting 
personnel  has  grown  significantly,  while  the  average  level  of  experience  has 
declined.  The  increase  in  workload  has  undoubtedly  resulted  in  less  time 
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being  available  to  process  individual  procurement  actions  and  also  suggests 
the  size  of  the  acquisition  workforce  is  inadequate.  (JAT,  2009,  p.  4). 

Over  the  last  ten  years,  the  military  has  seen  an  increase  in  the  percentage  of 
the  contracting  workforce  with  0-9  years  of  experience  and  20+  years  of  experience, 
while  there  has  been  a  decrease  in  the  percentage  of  the  group  with  10-19  years  of 
experience  (see  earlier  discussion).  The  middle  group  is  responsible  for  much  of  the 
source  selection  process,  and  it  is  the  smallest  group  proportionally  in  the  entire 
contracting  force.  It  is  possible  that  civilian  contracting  agencies  and  companies  are 
snatching  up  this  middle  group  as  they  come  to  the  end  of  their  initial  military 
commitment  by  offering  them  better  compensation  than  the  military  does.  This  is  a 
cause  for  concern  within  the  contracting  force  because  with  such  a  large  percentage 
of  workers  ready  for  retirement,  the  Defense  Department  could  conceivably  lose 
many  of  its  experienced  workers  within  a  short  time  period. 

Closely  related  to  the  problem  of  inadequate  manpower  is  inadequate  training 
on  source  selection  and  the  bid  protest  process.  “There  is  only  one  formal  DAU 
training  course  directly  focused  on  proper  source  selection  procedures.  This  course 
is  not  mandatory,  offered  only  online,  and  fails  to  capture  any  of  the  lessons  learned 
from  recent  GAO  protest  decisions”  (JAT,  2009,  p.  4).  This  lack  of  formal  training 
puts  Defense  Department  contracting  officials  at  a  disadvantage  from  the  start  of 
their  careers  because  if  they  are  not  trained  on  how  to  recognize  the  common 
contracting  pitfalls  that  often  lead  to  protests,  they  will  not  be  able  to  avoid  them  as 
easily  in  the  future  during  the  initial  stages  of  the  procurement  process — ultimately  at 
the  cost  of  future  DoD  time,  money,  and  manpower. 

A  final  contributor  to  the  increase  in  the  effectiveness  rate  is  the  loss  of 
opportunity  for  the  agency  to  learn  from  previous  procurement  mistakes  in  order  to 
improve  its  performance  in  the  future. 

There  is  currently  no  central  repository  for  protest  data  or  established  forums 
for  dialogue  on  protest  related  cases  and  trends  in  DoD.  The  GAO  does  not 
publish  detailed  information  on  cases  dismissed  after  voluntary  corrective 
action  or  disclose  the  specific  corrective  action  taken,  which  makes  it  very 
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difficult  to  ascertain  the  nature  of  the  mistakes  that  were  made  in  these 
procurements.  (JAT,  2009,  p.  4) 

In  the  1980s,  once  the  GAO  formally  issued  a  decision  resolving  a  protest  or 
an  agency  took  corrective  action,  the  GAO  would  write  Secretaries  of  agencies  with 
forward-looking  recommendations  on  improving  agency  procurements.  Further,  the 
GAO  would  compile  and  submit  those  recommendations  to  Congress  in  the  GAO’s 
annual  statutory  reports  on  bid  protests.94  However,  “the  GAO  over  time  abandoned 
its  statutory  responsibility  to  make  recommendations  promoting  compliance  with 
procurement  laws”  (Spriggs  &  Kidalov,  2003).  As  a  result,  neither  the  protesters,  nor 
the  Congress,  nor  the  leadership  of  the  Department  of  Defense  can  be  assured  that 
anti-competitive  procurement  practices,  mistakes,  and  instances  of  misconduct  are 
not  repeated. 

Restoring  the  GAO’s  statutory  responsibilities  on  procurement  improvement 
recommendations  and  creating  a  central  repository  for  information  on  bid  protests 
would  be  useful  in  order  to  ensure  that  current  contracting  officials  have  the  most  up- 
to-date  training  possible  by  performing  case  studies  of  protest-related  cases;  it 
would  also  be  very  helpful  for  instructors  in  a  protest-related  source  selection  course 
to  have  contracting  students  perform  case  studies  and  learn  from  past  mistakes  in 
order  to  ensure  better  future  performance. 


94  For  example,  the  GAO  Statutory  Report  on  FY1985  Bid  Protest  Activity  to  the  Honorable  George 
Bush,  President  of  the  Senate,  B-1 58766  (Jan.  31,  1986)  reported  that  in  two  bid  protests  corrective 
action  was  either  limited  or  not  available,  but  the  Secretary  of  the  Army  and  the  Secretary  of 
Agriculture  were  asked  to  take  steps  to  prevent  recurrence  of  violations  in  the  future.  Both 
Secretaries  agreed  with  these  recommendations.  As  a  result,  the  Department  of  Agriculture 
admonished  the  contracting  officer  as  to  equal  and  fair  treatment  of  offerors  in  best  value 
procurements,  and  the  Department  of  the  Army  took  apparently  procedural  steps  ensuring  that  sole 
source  contracts  are  not  awarded  prior  to  the  expiration  of  the  required  notice  and  waiting  period. 
These  remedies  apparently  vindicated  the  public  interest  without  causing  undue  burden  to  agency 
operations. 
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C.  Hypotheses 


After  analyzing  the  data  and  observing  trends,  seven  different  hypotheses 
were  formed  on  how  to  improve  the  bid  protest  process  in  order  to  save  the  DoD 
time,  money,  and  manpower. 

A  New  Paradigm  to  Address  Bid  protests  suggests  that  any  proposal  to 
reduce  the  rate  of  successful  protests  will  involve  either  reducing  the  expected 
benefits  of  filing  or  litigating  a  protest,  or  increasing  the  expected  costs  (or  some 
combination). 

First,  better — not  necessarily  more — training  for  contracting  officials  with  a 
focus  on  better  attention  to  detail  will  result  in  fewer  mistakes  when  soliciting 
contractors,  which  will  lead  to  fewer  protests. 

Second,  if  the  Defense  Department  offers  smaller,  more  frequent  contracts,  it 
will  more  often  lead  to  contractors  accepting  the  loss  of  an  award  as  a  learning 
experience;  they  will  better  prepare  for  future  contracts  instead  of  protesting  the  loss 
of  the  one  current  large  procurement,  a  loss  that  shuts  them  out  of  the  market  for  a 
long  time. 

Next,  better  tracking  of  protest  information  will  enable  researchers  and 
auditors  to  periodically  analyze  trends  and  assure  quality  control  of  the  procurement 
process. 

Also,  by  including  a  rating  on  companies  that  considers  frivolous  protest 
history  as  part  of  the  past  performance  rating  during  the  source  selection  process, 
the  DoD  could  influence  the  company’s  decision  to  protest  unnecessarily  in  the 
event  of  losing  a  bid  for  a  contract.  To  pass  constitutional  muster  and  meet  the 
principles  of  fundamental  fairness,  the  criteria  for  this  rating  must  strictly  conform  to 
the  standards  for  frivolous  protests  set  forth  in  Rule  1 1  of  the  Court  of  Federal 
Claims  and  in  judicial  precedents. 
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Strategically  manage/reveal/report  Settlements:  Government  agencies  often 
voluntarily  give  aid  to  protestors;  this  contributes  to  the  effectiveness  rate.  If  these 
agencies,  specifically  the  DoD,  stopped  giving  inappropriate  voluntary  aid  to 
protestors  before  the  GAO  decision,  they  could  reduce  the  spending  of  unnecessary 
resources  in  the  event  of  a  deny  decision.  On  the  other  hand,  if  agencies  took 
corrective  action  or  entered  into  ADR  without  unnecessary  delays  and  for  proper 
reasons,  agencies  would  avoid  disruptions  from  mandatory  stays  and  avoid 
needlessly  paying  the  protester’s  legal  expenses.  Congress  and  the  agencies 
should  consider  making  all  bid  protests  settlements  public. 

To  better  manage  the  protest  process,  agencies  must  require  their  acquisition 
managers  to  manage  the  protest  process  in  a  business-like  manner  by  evaluating 
cost-benefit  tradeoffs  and  decision  steps  in  an  appropriate  and  timely  manner. 

Finally,  ADR  should  be  made  the  default  dispute  resolution  mechanism  in  bid 
protests,  and  agencies  should  be  required  to  file  justifications  as  to  why  they  have 
not  successfully  used  ADR.  In  addition,  agencies  should  be  required  to  justify  their 
failure  to  use  stay  overrides  and  other  flexibilities  and  defenses  that  would  minimize 
the  time  and  cost  impacts  of  bid  protests. 
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D. 


Recommendations  to  Reduce  Risk  of  Protest 


Goals: 

1.  Minimize  fraud,  favoritism,  and  corruption  (build  integrity,  increase 
transparency,  and  improve  accountability) 

2.  Maximize  competition  (increase  the  expected  value  of  participation  in 
agency  procurement-solicitation  and  award  processes) 

a.  Assumption:  “As  the  number  of  bidders  rises,  competition 
increases  and  the  purchase  price  falls”  (MMR,  1991 ,  p.  3).  An 
exception  in  which  restricting  bidders  makes  sense  would  be  a 
case  in  which  an  agency  would  waste  resources  on  evaluating 
the  bids  of  clearly  non-performing  firms  or  of  those  with  clearly 
inferior  (or  ill-matched)  products.  TCE  suggests  that  if  the 
expected  surplus  (e.g.,  production  cost  savings)  that  a  firm 
could  offer  is  not  sufficient  to  cover  the  transaction  costs  of 
preparing  and  evaluating  a  bid,  then  it  should  be  excluded. 
(However,  the  contestable  markets  threat  of  entry  theory  by 
Baumol  et  al.  [1 982]  might  justify  including  some  of  these  firms.) 

b.  One  option  is  to  compensate  all  bidders  for  their  bid  preparation 
costs  (CB)  to  promote  participation  by  more  bidders  (since 
3V73CB  <  0).  In  many  complex  and  uncertain  RFPs,  bid 
preparation  costs  can  be  thought  of  as  analogous  to  research 
and  development  costs.  This  is  especially  suitable  for  small 
business  concerns  and  new  entrants  to  the  Federal 
procurement  market. 

c.  Another  option  is  to  not  only  compensate  the  winner  for 
production  costs  that  were  bid,  but  also  to  provide  the  winner 
with  a  profit  related  to  its  advantage  over  its  next-best  rival.  The 
profit  earned  by  the  winner  should  depend  on  the  gap  between 
its  bid  and  the  second-best  bidder  to  provide  an  incentive  for  the 
winner  to  bid  more  aggressively  and  to  provide  a  higher  value 
(surplus)  product. 

d.  Protests  can  reduce  the  winner’s  profit  by  the  costs  (C3) 
required  to  defend  against  a  protest  (which  might  include 
settlement  costs),  where  C3=f(E,  F,  CF,  C1,C2). 

There  is  scant  empirical  evidence  forjudging  what  net  effect  the  modern 
system  of  protest  controls,  including  the  CICA  and  related  protest  reforms,  has  had 
on  procurement  outcomes. 
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A  rent-seeking  argument  suggests  that  influence  activities  are  attenuated  by 
protests,  but  are  there  less  disruptive  alternatives?  The  protest  mechanism  reduces 
the  rents  from  influence  activities  in  two  ways.  First,  a  successful  protest  deprives 
the  influence-seeking  firm  of  a  contract  and  forces  it  to  pay  protest  costs.  Second,  a 
settlement  forces  the  winner  to  share  part  of  the  profit  from  improper  influence.  The 
reduction  in  expected  rents  causes  investment  in  influence  activities  to  fall,  thereby 
increasing  the  probability  of  an  efficient  award  and  reducing  wasteful  rent-seeking 
activities. 


E.  Costs  of  Protests 

1 .  Cost  and  opportunity  costs  of  protest  authorities;  and  costs  and 
opportunity  costs  of  collecting  and  analyzing  information. 

2.  Cost  and  opportunity  costs  of  resources  used  by  protesting  contractor, 
winning  contractor  and  agency  (prosecuting  and  defending  protest): 
measure  resources  dedicated  to  personnel  training,  source  selection, 
and  defense  of  protests. 

3.  Schedule  delays  (delivery  of  goods  and  services):  measure  time 
required  to  purchase  and  field  equipment  with  and  without  protests. 

4.  Reduced  productivity  and  performance:  measure  user  perception  of 
quality  and  performance  with  and  without  protests. 

5.  Bidders  charge  government  a  price  premium  to  compensate  for  risk  of 
protests  (either  having  to  initiate  a  protest  or  defend  against  one). 

6.  Some  government  officials  may  be  tacitly  and  improperly  using  the 
protest  process  to  “buy  a  seal  of  approval”  from  the  GAO  for  agencies’ 
procurement  strategies,  even  at  the  risk  of  delays  from  mandatory 
stays  and  unnecessary  legal  expenses. 

7.  Strategic  manipulation  of  protest  process  as  barrier  to  entry  (force  new 
entrants  to  face  higher  entry  costs  of  protest),  or  incumbent  can  delay 
contract  award  with  protest;  or  seek  information  to  increase 
competitive  advantage  in  subsequent  procurements. 

8.  Discourage  government  purchasing  officials  from  exercising 
inappropriate  discretion  and  considering  important  subjective  factors 
when  incomplete  contracts  cannot  include  all  contingencies  (past 
performance,  forced  to  rely  on  objective  criteria,  reveal  too  early  those 
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criteria  and  relative  weights,  reduces  subjective  evaluation  of  vendor 
responsiveness,  etc.).  TCE:  If  agencies  cannot  write  a  complete 
contract,  then  vendors  have  incentives  to  shirk  in  the  provision  of  those 
dimensions  that  are  undervalued  or  not  explicitly  requested  in  the  RFP. 

9.  Bid  rotation  (bid  rigging)  cartel  enforcement  mechanism.  “The  protest 
enables  cartel  participants  to  challenge  an  award  to  a  cartel  member 
who  [violates]  the  collusive  agreement  by  bidding  low  ‘out  of  turn.’  The 
settlement  process  can  facilitate  efforts  to  extract  side  payments, 
including  subcontracts  (“greenmail”). 

10.  Reducing  protest  risks  could  increase  competition.  Protests  are  bad  for 
winners  (they  reduce  expected  benefits)  and  good  for  losers  (they 
provide  a  second  chance).  Need  grievances  addressed  to  encourage 
participation,  but  also  impacts  the  likelihood  the  winner  will  engage 
since  it  increases  the  cost  of  the  transaction  (defending  against  a 
protest  and  delays).  The  net  effect  on  competition  (increasing  number 
of  bidders)  is  an  empirical  question:  attempt  to  assess  the  effect  of 
protests  on  the  number  and  quality  of  suppliers  and  on  resources 
spent  to  obtain  government  contracts  or  to  raise  funds  in  capital 
markets. 

NOTE:  The  empirically  small  fraction  of  protests  sustained  suggests  that 
agencies  could  more  often  appeal  to  urgent  and  compelling  to  override  an  automatic 


NOTE:  Increasing  protestor  costs  by  increasing  data  collection  (breadth  of 
evidence  gathering)  has  two  offsetting  effects.  On  the  one  hand,  it  increases  the  cost 
of  a  protest,  which  tends  to  reduce  incentives  to  protest.  But  on  the  other  hand,  it 
might  also  make  the  protest  more  valuable  by  increasing  the  ability  of  protestors  to 
obtain  information  suggesting  procurement  flaws  (increasing  the  probability  of  merit 
and  sustained  given  merit),  and  revealing  additional  information  that  may  be  useful 
in  future  competitions,  increasing  the  incentive  to  protest. 

Transaction  Cost  Economics  (TCE)  Hypothesis:  The  increased  risk  of 
protests  increases  transaction  costs.  (Winners  lose  and  losers  win:  Fraction  of 
Responsible  Winning  Bidders  opting  out  =  f(Fraction  of  Contracts  Protested)). 
Institutions  will  be  created  to  minimize  transaction  costs,  or  maximize  the  net  benefit 
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from  the  transaction.  What  are  characteristics  of  transactions  that  tend  to  draw 
protests? 

Probit/Logit:  Prob.  of  Protest  =  f(Complexity;  Uncertainty;  Frequency  (length 
of  contract;  sequential  contracts);  Asset  Specificity;  Size  of  Contract;  Purchase  (split 
buy);  Lease;  Fee-for-Service;  Alternative  Opportunity;  Settlement;  Subcontract;  etc.) 

F.  Recommendations 

Transaction  Costs  Guidance — 

1 .  Reduce  Asset  Specificity 

2.  Reduce  Complexity — unbundle  (invest  in  systems  integration) 

3.  Reduce  Uncertainty 

4.  Increase  Frequency 

5.  Increase  Contestability 

6.  Invest  in  Governance  Mechanisms  (Measurement,  Monitoring  & 

Oversight) 

Suggestions  (Note:  legal  and  regulatory  impacts  have  not  been  determined) 

Demand  Curves  slope  down:  Make  the  prosecution  of  protest  more 
difficult  (reduce  the  probability  of  merit  and  of  sustain  given  merit  (PM 
and  PS)  and  raise  the  costs  of  protests  to  reduce  the  volume  of 
protests  (CF,  Cl,  C2). 

Apply  a  more  deferential  standard  (a  presumption  of  agency 
correctness  that  removes  the  automatic  stay)  similar  to  the  standard 
for  injunctions  used  at  the  Court  of  Federal  Claims. 

Impose  sanctions  on  frivolous  protests  in  the  manner  of  Rule  1 1  of  the 
Court  of  Federal  Claims. 

Limit  the  number  of  protests  allowed  in  a  given  period  or  over  a  given 
set  of  contracts.  Limit  future  protests  and  protest  appeals  if  the  first 
protest  is  strongly  denied.  Again,  these  limitations  should  be  modeled 
on  traditional  Rule  1 1  sanctions  on  repeated  frivolous  protest  filings  at 
the  Court  of  Federal  Claims. 
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Restrict  time  limits  to  file  protests,  especially  to  ensure  that  post-award 
protesters  do  not  obtain  pre-award  remedies  of  total  recompetition. 

Increase  financial  costs  of  protests,  e.g.  by  requiring  bonds  to 
compensate  the  government  for  the  cost  of  stays  similarly  to  Rule  65  of 
the  Court  of  Federal  Claims. 

Loser  pays — reimburse  the  government  and  winning  bidder  for  the 
legal  fees  and  costs  associated  with  defending  the  purchasing 
decision,  including  any  costs  from  schedule  delays,  cost  overruns,  and 
performance  shortfalls  due  to  the  automatic  stay.  Make  stays 
discretionary  by  allowing  the  protestor  to  request  (or  not  request)  a 
stay  since  loser  pays  (C2). 

Amend  the  charter  of  the  Armed  Services  Board  of  Contract  Appeals 
by  granting  it  the  authority  to  administer  and  conduct  ADR  in  bid 
protests. 

Competition  among  rival  government  forums  (including  inter-agency 
competition)  can  improve  the  quality,  reduce  costs,  and  make  more 
alternatives  available  to  protestors. 

Limit  discovery  tools  (the  ability  of  protestors  to  gather  information) 
and/or  use  of  discovered  information  so  that  protestors  do  not  gain 
inappropriate  competitive  advantage  over  non-intervenors  in  future 
competitions. 

Limit  remedies  for  successful  protests  to  ensure  no  unfair  competitive 
advantage  and  promote  public  interest  (. XA ) 

Require  exhaustion  of  administrative  remedies  (agency-level  protests) 
before  allowing  GAO  protests. 

Preserve  the  perception  of  institutional  and  political  neutrality. 

Reduce  the  scope  of  protest  oversight  and  increase  the  use  of  more 
cost-effective  alternatives. 

Reduce  overhead  costs  and  defensive  medicine  costs  associated  with 
bid  protests. 

Make  it  easier  to  assess  the  probable  outcome  of  a  protest. 

Establish  a  streamlined  procedures  for  obtaining  Office  of  Federal 
Procurement  Policy  or  Department  of  Justice  support  for  disagreement 
with  the  GAO’s  protest  recommendations. 
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18.  Adoption  of  expeditious  ADR  as  the  default  mechanism  for  resolving 
bid  protests. 

19.  Increase  incentives  and  align  the  interests  of  government  procurement 
officials  so  that  they  invest  in  human  capital  and  effort  in  order  to 
improve  procurement  outcomes.  Requiring  agencies  to  justify  failure 
to  use  ADR,  stay  overrides,  or  other  protest  resolution  flexibilities 
should  be  the  first  step  in  this  direction. 

20.  Facilitate  pre-award  protests  in  order  to  reduce  post-award  protests, 
but  be  careful  what  you  wish  for.  This  could  lead  to  multiple  stays  and 
constant  interruption  and  delays  in  the  procurement  process. 

21 .  Impact  of  bid  protest  remedies  in  major  cases  should  be  subject  to  ex¬ 
post  evaluation  on  procurement  outcomes  (Office  of  Federal 
Procurement  Policy  or  GAO  Acquisition  and  Sourcing  Management). 

22.  Experiment  with  different  levels  of  protest  oversight  and  evaluate 
performance. 

a.  Competition:  Is  the  strength  of  protest  oversight  correlated  with 
the  willingness  of  buyers  to  do  business  with  the  government 
agency? 

b.  Fraud,  favoritism,  and  incompetence:  What  are  the  delays  and 
value  for  money  associated  with  protest  oversight? 

This  study  offers  several  important  insights  and  policy 
recommendations: 

A  major  challenge  for  the  Defense  Department  is  to  reap  the  well- 
intentioned  benefits  of  protests  while  limiting  their  the  unintended 
consequences  (costs  and  disruptions). 

The  costs  of  preventing  or  deterring  inappropriate  settlements  is  an 
additional  transaction  cost  not  clearly  understood  or  appreciated  in  the 
current  operation  of  the  protest  process.  Evaluate  the  risk  that  protest 
systems  might  inadvertently  motivate  inappropriate  settlements  or 
inappropriate  litigation  strategies  to  protect  procurement  actions. 

The  risk  of  Fed-mail  and  Buy-off  settlements  warrants  a  serious 
rethinking  by  the  GAO  of  its  protest  effectiveness  rate  measure.  It  also 
cautions  against  departments,  agencies,  or  Congress  implementing 
any  analysis  or  recommendations  that  utilize  this  measure. 
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The  data  suggests  that  it  may  be  appropriate  (a)  to  overturn  the  implicit 
assumption  that  procurement  officials  (PO)  are  guilty  until  proven 
innocent  that  underpins  the  GAO’s  current  automatic  stay  of  disputed 
procurements,  and  (b)  to  replace  it  with  the  assumption  that  POs  are 
innocent  until  proven  guilty.  Under  today’s  binding  regulations  and 
statutes,  this  argues  for  departments  and  agencies  to  restrict  stays  (or 
injunctions)  through  more  frequent  use  of  (stay  override)  arguments 
such  as  “urgent  and  compelling  circumstances”  or  “in  the  interest  of  the 
United  States.” 

From  the  point  of  view  of  a  protester,  the  economics  approach 
suggests  that  there  is  no  such  thing  as  a  frivolous  lawsuit.  A 
disappointed  bidder  will  only  file  a  protest  if  his  expected  benefits 
outweigh  his  expected  costs.  Frivolous  lawsuits  are  an  example  of  the 
strategic  use  of  protests  by  firms  when  the  reasons  for  protests  are 
inconsistent  with  the  public  interest. 

In  shaping  protest  policy,  senior  leadership  should  focus  its  attention 
on  critical  procurements — i.e.,  on  those  acquisitions  likely  to  have  the 
greatest  impacts  on  investments  and  operations  if  they  are  protested — 
and  possibly,  but  not  necessarily,  on  high-cost  MDAPs.  To  help  shape 
protest  policies,  focus  on  identifying  and  addressing  key  factors  and 
characteristics  associated  or  correlated  with  successful  protests  of 
critical,  high-impact  procurements. 

Balancing  the  benefits  of  protests  against  the  potentially  significant 
(opportunity  and  transaction)  costs  suggests  that  there  is  an  optimum 
rate  of  successful  protests. 

Regardless  of  where  the  Department  happens  to  be  relative  to  the 
optimum,  there  exist  policies  to  reduce  successful  protests  that  can 
actually  increase  competition. 

While  selected  policies  that  reduce  Defense  Department  protest  risks 
might  increase  risks  faced  by  losing  bidder  types,  hurting  competition, 
they  could  reduce  risks  faced  by  winning  bidder  types,  boosting 
competition.  The  net  effect  is  an  empirical  question  with  enormous 
policy  implications  that  warrants  an  empirical  study  to  investigate  this 
issue. 

Although  a  fear  generally  expressed  in  the  legal  and  regulatory 
literature  is  that  limiting  protests  will  inhibit  competition,  resulting  in 
higher  costs,  those  costs  could  be  more  than  offset  by  reductions  in 
delays  and  disruptions  from  fewer  protests. 
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The  challenge  is  to  choose  an  optimum  mix  of  governance 
mechanisms  that  minimizes  the  costs  of  aligning  incentives — to  reduce 
the  risk  of  fraud  and  errors  and  encourage  competition — and  that 
guarantees  the  best  possible  procurement  outcomes  for  our  troops  and 
taxpayers. 

Ideally,  policy  initiatives  to  limit  protests  could  be  combined  with  other 
carefully  selected,  cost-effective  governance  mechanisms  (internal 
audits,  external  audits,  independent  investigations,  training 
investments,  etc.)  that  reduce  the  need  for  the  monitoring  and 
oversight  role  of  protests. 

The  challenge  is  to  choose  the  most  cost-effective  mix  of  initiatives  that 
align  incentives  and  that  provide  adequate  monitoring  and  enforcement 
of  procurements  to  reduce  the  risk  of  fraud  and  errors,  and  to 
encourage  competition.  In  searching  for  alternative  approaches,  the 
optimum  mix  will  include  targeted  investments  in  human  capital  as  well 
as  in  centralized  (audits  and  investigations)  and  decentralized  (bid 
protests)  third-party  governance  mechanisms. 

Because  most  agency  protest  decisions  are  not  published,  there  is 
limited  transparency,  and  the  lack  of  a  record  of  precedent  deprives 
future  protesters  of  the  benefit  of  the  agency’s  reasoning  on  prior 
decisions.  It  also  deprives  agencies  of  the  opportunity  to  learn  from  the 
process  in  order  to  improve  their  processes  and  procedures.  A 
recommendation  is  for  agencies  to  publish  their  protest  decisions. 

GAO  should  again  make  recommendations  to  agencies  on  prospective 
improvements  in  their  procurement  processes  following  each  bid 
protest  where  the  protest  was  found  meritorious  or  subject  to  corrective 
actions.  GAO  should  include  those  recommendations,  as  well  as  the 
most  common  grounds  for  sustaining  protests,  in  its  annual  report  to 
Congress. 

A  recommendation  is  to  seek  a  broader  mix  of  reviewers  for  protests 
within  the  GAO  that  includes  subject  matter  experts  (SMEs)  and 
economists,  especially  to  assess  the  costs  and  benefits  of  delays  and 
remedies. 
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17.  The  operation  of  the  current  protest  system  reflects  an  assumption  that 
the  costs  of  protests  are  less  than  the  costs  of  relying  on  alternative 
monitoring  and  enforcement  mechanisms  that  would  achieve  the  same 
policy  goals  (integrity,  equity,  efficiency)  and  “ensure  the  proper 
functioning  of  the  procurement  system,  and  to  promote  confidence  in 
the  system.”  A  recommendation  is  to  evaluate  the  costs  and  benefits  of 
alternative  approaches  that  might  improve  procurement  outcomes — 
including  human  capital  investments  and  third-party  governance 
mechanisms. 
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VI. 


Recommendations  and  Areas  for  Further 
Research  from  the  Companion  Report, 
Better  Acquisition  Management  through 
Alternative  Dispute  Resolution  (ADR)  and 
Other  Best  Practices  for  Preventing  and 
Resolving  Bid  Protests 

A.  Recommendations 

1.  Case  for  Fiscal  Disincentives  beyond  Truly  Frivolous  Protests  is 
not  Demonstrated,  But  GAO  Rules  Must  Include  COFC-Style 
Sanctions 

Our  research  has  revealed  that  agency  personnel  have  strong  opinions 
regarding  the  need  for  some  type  of  disincentives  for  unsuccessful  protestors.  The 
opinions  range  from  “loser  pays”  scenarios,  where  the  losing  party  to  the  protest 
must  reimburse  the  other  for  the  cost  of  the  unsuccessful  litigation,  to  more  reserved 
opinions,  where  penalties  are  only  assessed  for  those  protests  that  are  deemed  to 
be  truly  frivolous. 

There  is  no  doubt  that  the  lack  of  disincentives  for  filing  an  unsuccessful 
protest  may  contribute  to  the  litigious  desires  of  a  would-be  protestor.  Nonetheless, 
we  are  not  convinced  that  such  disincentives  are  prudent,  Constitutional,  or  conform 
to  international  obligations  of  the  United  States.  While  such  a  procedure  may  very 
well  reduce  the  negative  impact  frivolous  protests  have  on  the  acquisition  system,  it 
may  do  so  at  the  expense  of  legitimate  protests.  Such  disincentives  may  actually 
discourage  participation  in  the  federal  contracting  process  and  as  a  result 
competition  may  decrease.  (GAO,  B-401197,  2009,  13).  Further,  in  order  to 
determine  whether  or  not  a  protest  is  indeed  frivolous,  it  would  require  additional 
action  on  the  part  of  the  GAO.  Currently,  GAO  determines  initially  whether  or  not  a 
protest  meets  the  requirements  for  filing  a  protest  (timely,  an  interested  party,  etc.) 
and  subsequently  determines  the  merit  of  the  protest.  In  order  to  determine  that  a 
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protest  is  sanctionable  as  frivolous,  GAO  would  have  to  make  a  separate 
determination.  This  new  determination  may  provide  deterrence,  but  at  the  cost  of 
additional  litigation.  Further,  unless  carefully  crafted,  such  disincentives  or  penalties 
will  likely  run  afoul  of  the  U.S.  Supreme  Court’s  First  Amendment  protections  for  the 
right  of  free  speech  and  the  right  to  petition  the  government  for  the  redress  of 
grievances,  or  the  Fifth  Amendment  right  to  due  process  of  law.  For  example,  the 
Supreme  Court  held  in  Bill  Johnson’s  Restaurants,  Inc.  v.  National  Labor  Relations 
Board  (461  U.S.  731  (1983)),  that  a  Federal  agency  cannot  halt  lawsuits  brought 
even  for  improper  motives  unless  those  lawsuits  are  based  on  “intentional 
falsehoods  or  on  knowingly  frivolous  claims,”  or  otherwise  lack  a  reasonable  basis. 

In  another  case,  California  Motor  Transport  Co.  v.  Trucking  Unlimited  (404  U.S.  508 
(1972)),  the  Supreme  Court  held  that  Federal  antitrust  laws  may  penalize 
businesses  bringing  lawsuits  and  petitions  to  Federal  agencies  only  if  such  petitions 
and  lawsuits  are  “a  mere  sham  to  cover  what  is  actually  nothing  more  than  an 
attempt  to  interfere  directly  with  a  business  relationship  of  a  competitor.”  Federal 
appellate  courts  also  identified  two  limited  ways  which  can  render  a  legal  action 
frivolous:  “First,  a  legal  action  is  considered  ‘frivolous  as  filed’  when  a  plaintiff  or 
appellant  grounds  its  case  on  arguments  or  issues  ‘that  are  beyond  the  reasonable 
contemplation  of  fair-minded  people,  and  no  basis  for  [the  party’s  position]  in  law  or 
fact  can  be  or  is  even  arguably  shown.’  .  .  .  Second,  a  legal  action  is  considered 
‘frivolous  as  argued’  when  a  plaintiff  or  appellant  has  not  dealt  fairly  with  the  court, 
has  significantly  misrepresented  the  law  or  facts,  or  has  abused  the  judicial  process 
by  repeatedly  litigating  the  same  issue  in  the  same  court.”  (GAO,  B-401 197,  2009, 

1 1 ).  Any  restriction,  disincentives,  or  penalties  for  strategic  protests  against  the 
public  interest  would  have  to  meet  the  Federal  courts’  stringent  requirements. 

Finally,  the  United  States  agreed  to  provide  effective  bid  protest  mechanisms  as  part 
of  the  World  Trade  Organization  Agreement  on  Government  Procurement  and 
agreed  to  effective  non-discrimination  in  procurement  as  part  of  Reciprocal  Defense 
Procurement  and  Acquisition  Policy  Memoranda  of  Understanding.  (Miller,  2009, 

93).  Care  must  be  taken  to  ensure  that  any  restrictions  conform  to  our  international 
trade  obligations  and  agreed-upon  standards  of  fairness. 
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The  U.S.  Court  of  Federal  Claims  provides  a  useful  model  for  sanctions 
against  improper  bid  protests.  Under  Rule  1 1  of  the  Rules  of  the  Court  of  Federal 
Claims,  a  protester  can  be  sanctioned  only  if  the  protest: 

(1 )  is  being  presented  for  any  improper  purpose,  such  as  to  harass, 
cause  unnecessary  delay,  or  needlessly  increase  the  cost  of  litigation; 

(2)  the  claims,  defenses,  and  other  legal  contentions  are  warranted  by 
existing  law  or  by  a  nonfrivolous  argument  for  extending,  modifying,  or 
reversing  existing  law  or  for  establishing  new  law; 

(3)  the  factual  contentions  have  evidentiary  support  or,  if  specifically  so 
identified,  will  likely  have  evidentiary  support  after  a  reasonable 
opportunity  for  further  investigation  or  discovery;  and 

(4)  the  denials  of  factual  contentions  are  warranted  on  the  evidence  or,  if 
specifically  so  identified,  are  reasonably  based  on  belief  or  a  lack  of 
information. 

Necessarily,  a  protestor  would  have  some  right  to  due  process  in  order  to 
“avoid  punishing  a  company  for  filing  a  good-faith  but  unmeritorious  protest.”  (GAO, 
B-401 197,  2009,  13).  This  process  may  actually  make  the  overall  protest  process 
longer,  which  could  have  potential  negative  impacts  on  the  acquisition  system. 
However,  bringing  the  GAO  Bid  Protest  Regulations  to  parity  with  COFC  concerning 
sanctions  for  truly  frivolous  protests  could  provide  a  deterrent  and,  more  importantly, 
address  the  perception  that  GAO  may  be  welcoming  frivolous  protests. 

2.  Standards  for  Mandatory  Stay  Overrides  in  CICA  and  Agency  FAR 
Supplements  Must  be  Clarified  and  Re-Balanced  to  Protect 
Taxpayers  and  the  Troops,  Not  Just  the  Protesters 

Our  research  demonstrates  that  contracting  agencies  must  be  much  more 
aggressive  at  issuing  and  defending  overrides  of  mandatory  CICA  stays  triggered  by 
filing  GAO  protests.  At  the  same  time,  our  research  shows  that  the  fault  does  not  lie 
entirely  with  the  agencies.  COFC  interpretations  of  what  CICA  requires  to  sustain 
an  override  seem  to  be  clearly  in  conflict  with  each  other  and  with  the  question  most 
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important  to  the  contracting  agency,  that  of  validity  of  contract  awards.  These  COFC 
interpretations  seem  to  lack  balance  between  the  interests  of  the  protesters  and  of 
the  taxpayers,  the  troops,  and  Federal  agencies.  Congress,  the  Comptroller 
General,  and  the  Federal  contracting  issues  should  clarify  and  strengthen  the  CICA 
override  standards. 

3.  Agency-Level  Protests  Must  Be  Made  More  Transparent  and 
Trustworthy  to  Be  Truly  Effective 

Our  research  reveals  that  experts  in  Federal  agencies  recognize  the  value  of 
agency-level  protests  as  a  speedier,  less  costly,  less  adversarial  alternative  to 
litigation.  The  Army  Materiel  Command  experience  is  particularly  instructive  on  this 
point.  However,  contractors  appear  to  distrust  the  agency-level  protest  process.  If 
Congress  and  the  Executive  Branch  are  serious  about  reducing  litigation  delays  and 
costs,  they  may  want  to  improve  the  transparency  and  trustworthiness  of  the 
agency-level  protest  process.  Specifically,  CICA  and  FAR  Part  33  may  be  amended 
to  provide  for  greater  disclosure  of  procurement  information,  mandatory  impartial 
review  above  the  contracting  officer  level,  publication  of  agency-level  protest 
decisions,  and  continuation  of  mandatory  stays  between  agency-level  protests  and 
any  subsequent  GAO  protests.  Once  agency-level  protests  earn  the  trust  of  the 
Federal  contracting  community,  it  may  be  worthwhile  to  require  contractors  to 
exhaust  their  agency-level  protest  remedies  before  filing  protests  with  the  GAO  or 
the  Court  of  Federal  Claims. 

4.  ADRA  Must  Be  Strengthened  to  Limit  Grounds  for  Refusing  ADR 

Our  research  reveals  that  at  least  some  Federal  agencies  may  be  tacitly 
condoning  or  encouraging  the  litigation  of  bid  protests  at  the  GAO.  While  ADRA 
allows  the  U.S.  government  to  decline  ADR  in  certain  circumstances,  our  research 
shows  that  contracting  agencies  are  litigating  bid  protests  far  more  often  than  even 
the  agencies  themselves  find  appropriate.  In  addition,  our  research  reveals  that 
agencies  go  along  with  GAO  protests  for  improper  reasons  not  authorized  under 
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ADRA.  Congress  should  take  steps  to  toughen  ADRA  in  order  to  ensure  that 
agencies  are  not  “buying”  GAO  decisions  with  taxpayer  funds. 

5.  ADR  Should  Be  the  Default  Dispute  Resolution  in  Federal 
Procurement  Protests 

Our  research  shows  that  the  Executive  Branch  has  extensive  experience  with 
ADR  as  the  default  bid  protest  resolution  system.  Originally  created  as  an 
exceptional  experiment,  the  protest  system  at  the  Federal  Aviation  Administration 
ODRA  demonstrated  its  ability  to  provide  significant  time  savings  and  achieve 
protest  resolution  in  a  less  disruptive  manner  than  the  GAO  protests.  Congress 
should  now  take  the  FAA  ODRA  lessons  and  replicate  them  across  the  Federal 
government. 

6.  Agencies  Must  Be  Required  to  Vigorously  Object  to  GAO  Protests 
when  Appropriate 

Related  to  the  matter  of  frivolous  protests  are  agency  actions  designed  to 
properly  document  their  objections  to  a  protest.  The  GAO  “dismisses  protests, 
where  appropriate,  without  the  need  to  resolve  whether  the  protest  was  frivolous.” 
(GAO,  B-401 197,  12).  It  is  incumbent  on  agencies  however,  to  object  to  protests 
when  appropriate.  Our  research  revealed  that  many  top  acquisition  and  legal 
professionals  believe  frivolous  protests  frequently  preclude  effective  resolution  of  bid 
protests.  Yet  the  very  same  individuals  indicated  that  they  believe  their  agencies 
infrequently  raise  vigorous  objections  in  response  to  frivolous  protests.  Indeed,  the 
last  recorded  “frivolous  protest”  objection  was  reported  by  the  GAO  back  in  1996. 
Since  the  GAO  will  dismiss  protests  that  are  frivolous  or  without  merit,  it  is 
incumbent  upon  agency  officials  to  raise  objections  to  protests  they  deem  to  be 
frivolous  or  otherwise  without  merit. 
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7.  Agencies  Must  Provide  Quality  Mandatory  and  Non-Mandatory 
Debriefings  to  Prevent  or  Limit  Strategic  Protests 

Numerous  survey  respondents  provided  comments  regarding  the  strategic 
reasons  that  may  induce  an  unsuccessful  offeror  to  file  a  protest.  One  respondent 
called  such  protests  “fishing  expeditions”  while  another  stated  it  was  a  means  to 
“circumvent  FOIA.”  Respondents’  rationale  is  that  by  filing  a  protest,  even  if 
ultimately  unsuccessful,  the  protestor  may  obtain  some  information  that  he  can  later 
use  to  his  advantage.  One  means  by  which  agencies  can  prevent  such  behavior  is 
to  provide  a  high  quality  debriefing,  regardless  of  whether  a  debriefing  is  mandatory. 
If  an  unsuccessful  offeror  has  been  thoroughly  debriefed  as  to  why  he  was  not 
selected  for  award,  he  may  not  feel  he  needs  any  additional  information,  and  as  a 
result  he  may  be  less  inclined  to  file  a  protest.  A  thorough  debrief  may  also 
convince  an  unsuccessful  offeror  that  he  was  in  fact  not  prejudiced  by  the  agency’s 
decision  and  therefore  has  no  need  to  file  a  protest. 

Even  with  complete  and  thorough  debriefings,  some  protestors  will  want  to  file 
a  protest  in  order  to  seek  information  not  otherwise  available  to  them.  A  change  in 
agency  policy  could  prevent  this  occurrence.  The  information  asymmetry  that  exists 
between  the  government  and  unsuccessful  offerors  may  very  well  influence  an 
offeror’s  decision  to  protest.  Government  agencies  can  prevent  this  from  occurring 
by  eliminating  or  reducing  the  asymmetry.  Information  that  is  normally  made 
available  to  an  unsuccessful  offeror  only  after  having  filed  a  protest  should  be  made 
available  to  the  offeror  absent  a  formal  protest  filing.  This  may  eliminate  one 
incentive  that  an  unsuccessful  offeror  has  in  filing  a  protest. 
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8. 


Defense  Agencies  Should  Object  to  GAO  Recommendations 
Whenever  Military  Needs  So  Require,  and  the  Office  of  Federal 
Procurement  Policy  and  the  Department  of  Justice  Should 
Streamline  the  Process  for  Supporting  Federal  Agencies  When 
the  GAO  Gets  It  Wrong 

Our  research  revealed  that  Federal  and  especially  defense  agencies  tend  to 
follow  the  GAO  blindly.  Significantly,  not  once  did  a  defense  agency  refuse  to  follow 
GAO  recommendations  in  order  field  equipment  to  the  troops.  Agencies  have 
traditionally  feared  Congressional  sanctions  for  refusing  to  follow  the  GAO,  but 
research  proves  that  this  fear  is  unwarranted.  Indeed,  our  research  shows  that 
during  the  2009-2010  dispute  between  the  GAO  and  the  Department  of  Defense 
concerning  the  latter’s  refusal  to  follow  the  former’s  protest  recommendations,  the 
Office  of  Management  and  Budget,  the  Department  of  Justice,  Congress,  and  the 
President  took  the  side  of  the  Department  of  Defense.  However,  the  process  for 
agencies  to  secure  high-level  support  against  wrong  GAO  recommendations  is  not 
well-established  within  the  Executive  Branch.  The  Federal  Acquisition  Regulation 
Council  should  make  appropriate  amendments. 

9.  CICA  and  GAO  Regulations  Should  Require  Protesters  to  Post 
Bonds  for  the  Cost  of  Interruption  from  CICA  Stays 

Our  review  shows  Federal  agencies  may  not  have  sufficient  tools  to  reduce 
costs  and  delays  from  mandatory  CICA  stays.  On  the  contrary,  under  Rule  65  of  the 
Court  of  Federal  Claims,  “[t]he  court  may  issue  a  preliminary  injunction  or  a 
temporary  restraining  order  only  if  the  movant  gives  security  in  an  amount  that  the 
court  considers  proper  to  pay  the  costs  and  damages  sustained  by  any  party  found 
to  have  been  wrongfully  enjoined  or  restrained.”  Further,  under  COFC  Rule  65.1 , 
“[t]he  surety’s  liability  may  be  enforced  on  motion  without  an  independent  action.” 
Congress  or  the  GAO  should  introduce  similar  rules  in  GAO  bid  protest  in  order  to 
compensate  the  taxpayers  for  the  costs  of  at  least  some  protests  that  are  ultimately 
found  to  lack  merit. 
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10.  Agency  Lawyers  and  Acquisition  Workforce  Must  Be  Trained  to 
Resolve  Bid  Protests  in  Accordance  with  Least  Disruptive  Paths 

Our  research  reveals  that  opportunities  abound  to  utilize  ADR  and  other 
dispute  resolution  flexibilities  even  under  current  GAO  Regulations,  COFC  Rules, 
and  agency-level  protest  rules  in  FAR  Part  33.  Unfortunately,  our  research  also 
reveals  that  agency  rarely  utilize  all  the  tools  available  to  reduce  delays  and  costs  of 
bid  protests.  In  the  beginning  of  this  paper,  we  showed  that  it  is  possible  to  map  out 
with  great  detail  the  least  disruptive  and  most  cost  effective  paths  for  resolution  of 
bid  protests,  as  well  as  related  decision  points.  Accordingly,  we  demonstrated  that 
Federal  contracting  officials  can  manage  the  bid  protest  process  in  the  same 
business-like  manner  as  they  manage  other  programs.  However,  our  survey  data 
reveals  significant  gaps  in  the  understanding  of  ADR  and  other  flexibility  tools  by  top 
Federal  acquisition  experts.  Therefore,  it  is  imperative  that  agency  protest  resolution 
procedures  and  related  training  programs  for  the  acquisition  workforce  be  revised 
and  improved  in  order  to  help  contracting  officers  master  the  least  disruptive  and 
most  cost  effective  resolution  paths.  This  is  training  is  particularly  important  in 
military  agencies,  which  are  characterized  by  lower  incidents  of  negotiation  and 
other  types  of  ADR  than  civilian  agencies. 

11.  To  Reduce  Protests,  Federal  Agencies  Should  Design  Acquisition 
Strategies  Promoting  Maximum  Possible  Competition 

Our  research  shows  that  some  of  the  most  effective  ways  to  prevent  bid 
protests  is  to  design  competitive  acquisition  strategies.  These  strategies  involve 
breaking  up  large  consolidated  contracts  into  smaller  contracts,  including  the 
awarding  of  multiple-award  contracts  instead  of  single-award  contracts.  The 
substantive  provisions  of  the  Competition  in  Contracting  Act  at  Title  10,  Section 
2304,  the  Small  Business  Act  at  Title  15,  Section  644,  the  anti-consolidation 
legislation  in  Title  10,  Section  2382,  and  FAR  Part  16.5  all  compel  emphasis  on  such 
competitive  contracts.  Now,  Federal  agencies  must  swiftly  implement  these  statutes 
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not  only  because  that  is  required,  but  because  doing  so  makes  business  sense  and 
compensate  the  taxpayers  through  reduced  protests  and  more  efficiency. 

12.  Federal  Regulations  Should  Require  Parties  and  Agency 

Acquisition  Officials  to  Justify  and  Document  Decisions  not  to 
Initiate  or  Use  ADR,  Stay  Overrides,  Express  Options,  Sanctions 
for  Truly  Frivolous  Filings,  and  other  Similar  Tools 

Our  review  of  the  Federal  Aviation  Administration  ODRA  procedures 
highlighted  the  primary  use  of  ADR  to  resolve  protests.  We  noted  the  fact  that 
ODRA  was  able  to  resolve  protests  more  quickly  than  when  compared  to  the  GAO’s 
bid  protest  process.  Within  ODRA,  there  is  an  overriding  presumption  that  parties 
will  resolve  protests  through  ADR.  If  unable  to  utilize  ADR,  both  parties  must  submit 
explanatory  statements  to  the  Dispute  Resolution  Officer.  The  presumption  that 
parties  will  utilize  ADR,  unless  there  is  a  countervailing  reason  to  the  contrary,  is  a 
primary  driver  behind  ODRA’s  processes.  The  GAO  in  its  regulations  and  the 
Executive  Branch  through  the  Federal  Acquisition  Regulation  or  agency 
supplements  should  adopt  a  similar  procedural  policy,  and  require  each  party  to 
conduct  cost-benefit  analysis  and  affirmatively  demonstrate  why  ADR  cannot  be 
utilized  to  resolve  the  protest.  Such  a  policy  would  likely  reduce  the  time  delays  and 
costs  an  agency  must  endure  during  a  bid  protest.  Agencies  should  also  be 
required  to  formally  demonstrate  why  it  does  not  seek  dismissal  as  frivolous  or 
meritless,  sanctions,  early  corrective  action,  or  stay  overrides.  Any  such  justification 
documents  must  include  business  analysis  that  the  costs  of  formal  litigation  under 
the  regular  process  at  the  GAO  and  compliance  with  the  GAO  recommendations 
would  outweigh  the  benefits  of  ADR  and/or  expedited  procedures.  The  analysis 
should  take  into  account  potential  for  disruption  that  can  be  created  when  a  post¬ 
award  protester  may  seek  a  pre-award  remedy  of  cancellation  and  change  in  the 
acquisition  strategy.  Agencies  should  be  proactive  in  initiating  ADR  and  not  wait 
until  the  GAO’s  negotiation  assistance  or  outcome  prediction. 
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B.  Areas  for  Further  Research 


Our  research  has  revealed  a  number  of  further  areas  of  study  that  warrant 
additional  research.  Specific  areas  of  study  include: 


Analyze  and  document  contractor  motivation  to  file  a  protest.  Such 
research  should  include  a  thorough  understanding  of  the  business 
decisions  that  either  constrain  or  enhance  a  contractor’s  motivation  to 
file  a  protest. 

Conduct  further  analysis  into  debriefing  procedures.  Best  practices  for 
conducting  a  successful  debriefing  need  to  be  researched  and 
documented. 

Research  into  areas  of  strategy  and  practices  that  survey  respondents 
cited  with  minimal  frequency.  A  number  of  these  strategies  and 
practices  appear  useful  in  mitigating  the  negative  effect  of  reducing  the 
impact  of  bid  protests,  yet  they  are  underreported  in  our  research. 
Further  study  into  this  area  should  be  conducted. 

Acquisition  planning  is  cited  the  most  commonly  cited  strategy  or 
practice  to  minimize  the  impact  of  bid  protests  on  the  acquisition 
process.  Best  practices  for  all  facets  of  acquisition  planning  should  be 
researched  and  documented. 

The  lack  of  financial  disincentives  for  unsuccessful  offerors  was  a 
highly  cited  policy  that  precludes  the  effective  resolution  or  avoidance 
of  bid  protests.  Although  we  have  previously  stated  our 
recommendation  on  this  policy,  we  nonetheless  feel  that  further 
research  is  warranted.  Specifically,  research  must  be  conducted  into 
the  negative  externalities  that  may  be  associated  with  financial 
disincentives. 
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